Text recognition method and device

Abstract

The present application provides a method and device for text recognition, the method comprising: correcting the direction of text in an input image to obtain a candidate image, wherein the input image is a building image; obtaining a feature map of the candidate image; enhancing the feature map based on the context information of the text to obtain a target feature map; performing text recognition based on the target feature map to obtain a text recognition result of the input image. The method in the embodiment of the present application can improve the accuracy of text recognition.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and device for text recognition. Background Art

[0002] With the rapid development of artificial intelligence, image recognition technology has been widely used in various fields, and optical character recognition (OCR), as an important application direction, has also received more and more attention.

[0003] At present, the architectural design industry usually uses computer aided design (CAD) software to draw architectural design drawings. The finished architectural design drawings need to be reviewed to determine whether they violate national standards. At present, experienced engineers are mainly relied on to review architectural design drawings, but the review task has a large workload and low efficiency. Therefore, it is urgent to use computers to replace manual work for automated review.

[0004] Architectural design drawings usually include various texts such as drawing names, description information, and drawing numbers. In order to realize automatic drawing review, it is necessary to accurately detect the text in architectural design drawings. However, architectural design drawings often include many lines and components, which will affect the recognition of their texts, resulting in low accuracy of text recognition. Summary of the invention

[0005] In view of this, the embodiments of the present application are directed to providing a method and apparatus for text recognition, which can improve the accuracy of text recognition.

[0006] In a first aspect, a method for text recognition is provided, the method comprising: correcting the direction of text in an input image to obtain a candidate image, wherein the input image is a building image; acquiring a feature map of the candidate image; performing feature enhancement on the feature map based on context information of the text to obtain a target feature map; performing text recognition based on the target feature map to obtain a text recognition result of the input image.

[0007] In an embodiment of the present application, the direction of the text in the input image is corrected so that the direction of the text in the input image can be made consistent, which can effectively reduce the difficulty of text recognition. Feature enhancement based on the contextual information of the text can improve the robustness of the text features in the target feature map. At this time, text recognition based on the target feature map can improve the accuracy of text recognition.

[0008] Optionally, the correcting the direction of the text in the input image to obtain the candidate image includes: performing semantic segmentation on the input image to obtain a semantic segmentation map, wherein the semantic segmentation map is used to indicate the text in the input image; performing text detection on the input image based on the semantic segmentation map to obtain a text detection result, wherein the text detection result is used to indicate the text area in the input image; and correcting the direction of the text based on the text detection result to obtain the candidate image.

[0009] Optionally, the directions of the texts in the candidate images are consistent.

[0010] Optionally, the feature map is enhanced based on the context information of the text to obtain a target feature map, including: using a deep learning model to enhance the feature map based on the context information of the text to obtain a target feature map, the deep learning model includes a bidirectional model structure composed of an encoder, a decoder and an attention mechanism.

[0011] In an embodiment of the present application, a deep learning model is used to enhance the feature map based on the context information of the text, so that the context information (such as the association or dependency features between characters) can be integrated into the target feature map, reducing background interference in the input image, and improving the robustness of text features in the target feature map. At this time, text recognition based on the target feature map can improve the accuracy of text recognition.

[0012] Optionally, the deep learning model can be a bidirectional transformer model.

[0013] Optionally, the use of the deep learning model to enhance the feature map based on the context information of the text to obtain a target feature map includes: using a feature pyramid network to convolve the feature map to obtain a multi-scale feature map; using the deep learning model to enhance the multi-scale feature map based on the context information of the text to obtain a target feature map.

[0014] In an embodiment of the present application, the multi-scale feature map is determined using a feature pyramid network, which can improve the receptive field of the model (i.e., the text recognition model). At this time, the multi-scale feature map is enhanced using the deep learning model to obtain a target feature map, and text recognition is performed based on the target feature map, which can improve the accuracy of text recognition.

[0015] Optionally, performing text recognition based on the target feature map to obtain a text recognition result of the input image includes: performing text recognition based on the target feature map using a connection temporal classification (CTC) model to obtain the text recognition result.

[0016] Optionally, obtaining the feature map of the candidate image includes: extracting the feature map of the candidate image using a lightweight network.

[0017] In an embodiment of the present application, a lightweight network is used to extract the feature map of the candidate image, which can improve the running speed of the model (ie, the text recognition model), thereby improving the efficiency of text recognition.

[0018] Optionally, the lightweight network consists of a separation convolution, a bottleneck layer including a 3x3 convolution, and a pooling layer.

[0019] In a second aspect, a device for text recognition is provided, comprising: a text detection unit, configured to perform text detection on an input image to obtain a text detection result, wherein the text detection result is used to indicate a text area in the input image, wherein the input image is a building image; a correction unit, configured to correct a direction of the text based on the text detection result to obtain a candidate image, wherein the input image is a building image; an acquisition unit, configured to obtain a feature map of the candidate image; a feature enhancement unit, configured to perform feature enhancement on the feature map based on context information of the text to obtain a target feature map; and a text recognition unit, configured to perform text recognition based on the target feature map to obtain a text recognition result of the input image.

[0020] In a third aspect, the present invention provides a device for text recognition, wherein the device is used to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0021] In a fourth aspect, a device for text recognition is provided, the device comprising a storage medium and a processor, the storage medium may be a non-volatile storage medium, the storage medium stores a computer executable program, the processor is connected to the non-volatile storage medium, and executes the computer executable program to implement the method in the first aspect or any possible implementation manner of the first aspect.

[0022] In a fifth aspect, a chip is provided, comprising a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to execute the method in the first aspect or any possible implementation of the first aspect.

[0023] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the first aspect or any possible implementation method of the first aspect.

[0024] In a sixth aspect, a computer-readable storage medium is provided, wherein the computer-readable medium stores a program code for execution by a device, wherein the program code includes instructions for executing the method in the first aspect or any possible implementation of the first aspect.

[0025] In an embodiment of the present application, the direction of the text in the input image is corrected so that the direction of the text in the input image can be made consistent, which can effectively reduce the difficulty of text recognition. Feature enhancement based on the contextual information of the text can improve the robustness of the text features in the target feature map. At this time, text recognition based on the target feature map can improve the accuracy of text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is an application scenario diagram applicable to an embodiment of the present application.

[0027] Figure 2 A schematic block diagram of a text recognition method in one embodiment of the present application.

[0028] Figure 3 A schematic block diagram of a text recognition method in another embodiment of the present application.

[0029] Figure 4 A structural diagram of a text recognition model in one embodiment of the present application.

[0030] Figure 5 The present invention is a schematic block diagram of a text recognition device in one embodiment of the present application.

[0031] Figure 6 The present invention is a schematic block diagram of a text recognition device in another embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The method in the embodiment of the present application can be applied to various scenarios of image processing for an image to be processed, which is not limited in the embodiment of the present application. For example, the method in the embodiment of the present application can be applied to a scenario of text recognition for an architectural image.

[0034] Figure 1 It is an application scenario diagram applicable to an embodiment of the present application. Figure 1The application scenario 100 may include an image to be processed 110 and an image processing device 120 .

[0035] It should be noted that Figure 1 The application scenarios shown are only examples and not limitations. Figure 1 The application scenario shown may include more or fewer devices or equipment, which is not limited in the embodiments of the present application.

[0036] Among them, the image to be processed 110 can be an architectural design drawing obtained based on computer aided design (CAD), and the image to be processed 110 can include one or more characters, which can be used to represent the drawing name, description information, drawing number and other contents. The embodiment of the present application does not limit the number or type of characters contained in the image to be processed 110.

[0037] For ease of description, in the embodiment of the present application, phrases or sentences consisting of text in the image 110 to be processed can be collectively referred to as text (or text lines). In the embodiment of the present application, the number of texts contained in the image 110 to be processed is not limited.

[0038] For example, the image to be processed 110 may be an architectural image drawn using AutoCAD software (or may also be referred to as an architectural design image), and the image to be processed 100 may include a drawing name, description information, drawing number, or other description or explanatory text. Alternatively, the image to be processed 110 may also be an architectural image drawn using other methods or other CAD software, and the type and format of the architectural image are not limited in the embodiments of the present application.

[0039] The image processing device 120 may be a computer device, a server (eg, a cloud server), or other devices or equipment capable of performing image processing (eg, performing text recognition based on a building image) on an image to be processed.

[0040] For example, the image processing device 120 may be a computer device, and the computer device may be a general-purpose computer or a computer device composed of a dedicated integrated circuit, etc., which is not limited in the embodiments of the present application.

[0041] Those skilled in the art will appreciate that the number of the above-mentioned computer devices may be one or more than one, and the types of the multiple computer devices may be the same or different. The number and device types of the terminals are not limited in the embodiments of the present application.

[0042] A neural network model may be deployed in the computer device to perform image processing on the image to be processed and obtain an image processing result indicating the image to be processed.

[0043] For example, the computer device may perform text recognition on a building image (i.e., the image to be processed) through a neural network model deployed therein (for example, the neural network model may be a text recognition model) to obtain a text recognition result of the building image.

[0044] The computer device may be a server or a cloud server, and directly performs image processing on the image to be processed.

[0045] Alternatively, the computer device may communicate with the server ( Figure 1 The computer device can send its to-be-processed image to the server, use the neural network model in the server to process the to-be-processed image, and store the obtained image processing result (such as the text recognition result of the to-be-processed image) as a sample image to train the neural network model in the server to obtain a neural network model for image processing.

[0046] The computer device can also obtain the image to be processed from the server, and then perform image processing on the image to be processed through the neural network model to obtain the image processing result of the image to be processed (such as the text recognition result of the image to be processed). The specific implementation method is not limited in the embodiment of the present application.

[0047] Usually, architectural design drawings include various texts such as drawing names, description information, and drawing numbers. In order to realize automated drawing review, it is necessary to accurately detect the text in architectural design drawings. However, the background in architectural design drawings greatly interferes with text recognition. For example, architectural design drawings often include many lines and components, and there will be overlaps between text, lines, components, and text. In addition, there are many styles of text (for example, text in various directions will appear in architectural design drawings). These will affect the recognition of text in architectural design drawings, resulting in low accuracy of text recognition.

[0048] Based on the above problems, a method for text recognition is proposed in an embodiment of the present application, which can improve the accuracy of text recognition.

[0049] Figure 2 is a schematic block diagram of a method 200 for text recognition according to an embodiment of the present application. The method 200 may be Figure 1 The image processing device 120 in the embodiment is executed, it should be understood that Figure 2 The steps or operations of method 200 are shown, but these steps or operations are only examples. The embodiment of the present application may also perform other operations or Figure 2 The method 200 may include steps S210 to S240, which are as follows:

[0050] S210, correcting the direction of the text in the input image to obtain a candidate image.

[0051] The input image can be Figure 1 The image to be processed 110 in the input image. Optionally, the input image may be a building image (or may also be called a building design drawing), and the input image may include one or more characters.

[0052] Optionally, the one or more characters may also form one or more texts (or may also be referred to as text lines). In the embodiments of the present application, there is no limitation on the arrangement of the characters in the text. For example, the characters in the text in the input image may be arranged in a straight line, or in a curve, or in any other line or form.

[0053] Optionally, the text included in the input image may have different directions. For example, the input image may include vertical (90 degrees or 270 degrees) text, inverted (180 degrees) text, or text in any other direction (or angle).

[0054] In some embodiments, correcting the directions of texts in the input image refers to making the directions of the texts in the input image consistent, that is, the directions of the texts in the candidate images obtained after correction are consistent.

[0055] For example, the direction of the text in the input image may be corrected to be horizontal (0 degrees). Of course, the direction of the text in the input image may also be corrected to any other direction (or any angle), which is not limited in the embodiments of the present application.

[0056] Furthermore, the direction of the text in the input image may also be corrected. For example, the text in the input image may be subjected to perspective change (or perspective deformation) to correct the direction (or angle) of the text in the text. The specific method of perspective change (or perspective deformation) can refer to the prior art, and the embodiments of the present application will not be described in detail. Of course, other methods may also be used to correct the direction of the text, and the embodiments of the present application will not be described in detail.

[0057] Optionally, the directions of the characters in the texts in the input image may be corrected so that the directions (or angles) of the characters in each text in the input image are consistent.

[0058] In some embodiments, in S210, semantic segmentation may be performed on the input image to obtain a semantic segmentation map; text detection may be performed on the input image based on the semantic segmentation map to obtain a text detection result; and the direction of the text may be corrected based on the text detection result to obtain the candidate image.

[0059] The semantic segmentation map may be used to indicate text in the input image, and the text detection result may be used to indicate a text area in the input image.

[0060] For example, a semantic segmentation model can be used to perform semantic segmentation on the input image to obtain a semantic segmentation map, which can be converted into a binary map based on a preset threshold (for example, elements in the semantic segmentation map that are greater than or equal to the threshold can be set to 1, and elements that are less than the threshold can be set to 0, and the specific method is not limited). Subsequently, an aggregation operation (such as a connected domain operation, etc.) can be used to convert the binary map into a text detection result. Finally, the direction of the text in the input image can be corrected based on the text detection result to obtain a candidate image.

[0061] S220: Obtain a feature map of the candidate image.

[0062] In some embodiments, a lightweight network may be used to extract the feature map of the candidate image, so that the running speed of the model (i.e., the text recognition model) can be increased, thereby improving the efficiency of text recognition.

[0063] Optionally, the lightweight network may be composed of a separation convolution, a bottleneck layer and a pooling layer, and the bottleneck layer may include a 3x3 convolution.

[0064] For example, the lightweight network may be a ShuffleNetV2 network. Of course, other lightweight networks may also be used in the embodiments of the present application, and the embodiments of the present application are not limited thereto.

[0065] For example, Figure 4 As shown, the first 4 layers of the ShuffleNetV2 network can be selected as the backbone network, and the layer corresponding to the second block of the first stage in the ShuffleNetV2 network (stage1 / block2 layer) can be used as the base layer to extract the feature map of the candidate image. Figure 3 The above method 300 is described in detail and will not be repeated here.

[0066] S230, performing feature enhancement on the feature map based on the context information of the text to obtain a target feature map.

[0067] In some embodiments, a deep learning model may be used in S230 to enhance the feature map based on the context information of the text to obtain a target feature map.

[0068] In an embodiment of the present application, a deep learning model is used to enhance the feature map based on the context information of the text, so that the context information (such as the association or dependency features between characters) can be integrated into the target feature map, reducing background interference in the input image, and improving the robustness of text features in the target feature map. At this time, text recognition based on the target feature map can improve the accuracy of text recognition.

[0069] Among them, the deep learning model may include a bidirectional model structure consisting of an encoder, a decoder and an attention mechanism.

[0070] For example, the deep learning model can be a bidirectional transformer model.

[0071] Of course, the deep learning model may also be other models that can extract contextual information of each text in the input image, which is not limited in the embodiments of the present application.

[0072] In some embodiments, before S230, feature pyramid networks (FPN) may be used to convolve the feature map to obtain a multi-scale feature map, and then the deep learning model may be used to enhance the multi-scale feature map based on the context information of the text to obtain a target feature map.

[0073] For example, Figure 4 As shown, the feature pyramid network may include 4 layers (4 convolutional layers), the number of channels of the 4 layers may be 32, 64, 128, and 256 respectively, and the step lengths of the 4 layers may all be 3*3. Figure 3 The above method 300 is described in detail and will not be repeated here.

[0074] In an embodiment of the present application, the multi-scale feature map is determined using a feature pyramid network, which can improve the receptive field of the model (i.e., the text recognition model). At this time, the multi-scale feature map is enhanced using the deep learning model to obtain a target feature map, and text recognition is performed based on the target feature map, which can improve the accuracy of text recognition.

[0075] S240, performing text recognition based on the target feature map to obtain a text recognition result of the input image.

[0076] In some embodiments, a connectionist temporal classification (CTC) model can be used to perform text recognition based on the target feature map to obtain the text recognition result. The specific text recognition method can refer to the prior art and will not be described in detail in the embodiments of this application.

[0077] In an embodiment of the present application, the direction of the text in the input image is corrected so that the direction of the text in the input image can be made consistent, which can effectively reduce the difficulty of text recognition. Feature enhancement based on the contextual information of the text can improve the robustness of the text features in the target feature map. At this time, text recognition based on the target feature map can improve the accuracy of text recognition.

[0078] Combine the following Figure 3 ,by Figure 4 Taking a specific text recognition model shown as an example, the method of text recognition in the embodiment of the present application is described.

[0079] Figure 3 is a schematic block diagram of a method 300 for text recognition according to an embodiment of the present application. The method 300 may be Figure 1 The image processing device 120 in the embodiment is executed, it should be understood that Figure 3 The steps or operations of method 300 are shown, but these steps or operations are only examples. The embodiment of the present application may also perform other operations or Figure 3 The method 300 may include steps S310 to S350, which are as follows:

[0080] S310, performing text detection on the input image to obtain a text detection result.

[0081] The input image may be a building image (or may also be called a building design drawing), and the input image may include one or more characters.

[0082] Optionally, the one or more characters may also form one or more texts (or may also be referred to as text lines). In the embodiments of the present application, there is no limitation on the arrangement of the characters in the text. For example, the characters in the text in the input image may be arranged in a straight line, or in a curve, or in any other line or form.

[0083] Optionally, the text included in the input image may have different directions. For example, the input image may include vertical (90 degrees or 270 degrees) text, inverted (180 degrees) text, or text in any other direction (or angle).

[0084] In some embodiments, semantic segmentation may be performed on the input image to obtain a semantic segmentation map; and text detection may be performed on the input image based on the semantic segmentation map to obtain a text detection result.

[0085] The semantic segmentation map may be used to indicate text in the input image, and the text detection result may be used to indicate a text area in the input image.

[0086] For example, the input image can be semantically segmented to obtain a semantic segmentation map, which can be converted into a binary map based on a preset threshold (for example, elements in the semantic segmentation map that are greater than or equal to the threshold can be set to 1, and elements that are less than the threshold can be set to 0, and the specific method is not limited). Subsequently, an aggregation operation (such as a connected domain operation, etc.) can be used to convert the binary map into a text detection result.

[0087] S320: Correct the direction of the text in the input image based on the text detection result to obtain a candidate image.

[0088] In some embodiments, correcting the directions of texts in the input image refers to making the directions of the texts in the input image consistent, that is, the directions of the texts in the candidate images obtained after correction are consistent.

[0089] For example, the direction of the text in the input image may be corrected to be horizontal (0 degrees). Of course, the direction of the text in the input image may also be corrected to any other direction (or any angle), which is not limited in the embodiments of the present application.

[0090] Furthermore, the direction of the text in the input image may also be corrected. For example, the text in the input image may be subjected to perspective change (or perspective deformation) to correct the direction (or angle) of the text in the text. The specific method of perspective change (or perspective deformation) can refer to the prior art, and the embodiments of the present application will not be described in detail. Of course, other methods may also be used to correct the direction of the text, and the embodiments of the present application will not be described in detail.

[0091] Optionally, the directions of the characters in the texts in the input image may be corrected so that the directions (or angles) of the characters in each text in the input image are consistent.

[0092] It should be noted that the S310 and S320 may also be Figure 1 or, optionally, the S310 and S320 may also be performed by a unit or module in a text recognition model, that is, the text recognition model in the embodiment of the present application may also include a unit or module for performing S310 and S320 ( Figure 4 (not shown in the text recognition model shown).

[0093] S330: Perform text recognition based on the candidate image to obtain a text recognition result of the input image.

[0094] In some embodiments, the Figure 4 The text recognition model shown performs text recognition on the candidate image to obtain the text recognition result.

[0095] Optionally, a lightweight network may be used to extract the feature map of the candidate image. In this way, the running speed of the model (i.e., the text recognition model) may be increased, thereby improving the efficiency of text recognition.

[0096] For example, Figure 4 As shown, the first 4 layers of the ShuffleNetV2 network can be selected as the backbone network, and the layer corresponding to the second block of the first stage in the ShuffleNetV2 network (stage1 / block2 layer) can be used as the base layer to extract the feature map of the candidate image. Figure 3 The above method 300 is described in detail and will not be repeated here.

[0097] Optionally, a feature pyramid network may be used to convolve the feature map to obtain a multi-scale feature map.

[0098] For example, Figure 4 As shown, the feature pyramid network may include 4 layers (4 convolutional layers), the number of channels of these 4 layers may be 32, 64, 128, and 256 respectively, and the step size of these 4 layers may all be 3*3.

[0099] Optionally, a deep learning model may be used to enhance the feature map based on contextual information of the text to obtain a target feature map.

[0100] The deep learning model may include a bidirectional model structure consisting of an encoder, a decoder and an attention mechanism. Optionally, the deep learning model may be a bidirectional transformer model.

[0101] For example, Figure 4As shown, the multi-scale feature maps can be first fused (for example, concatenated), and then convolved with a step size of 1*1 and a channel number of 256. Subsequently, the bidirectional transformer model is used to enhance the features of the result (after concat and convolution) to obtain the target feature map.

[0102] Optionally, a recognition model may be used to perform text recognition based on the target feature map to obtain the text recognition result.

[0103] For example, Figure 4 As shown, the CTC model can be used to perform text recognition on the target feature map to obtain the text recognition result.

[0104] In an embodiment of the present application, the direction of the text in the input image is corrected so that the direction of the text in the input image can be made consistent, which can effectively reduce the difficulty of text recognition. Feature enhancement based on the contextual information of the text can improve the robustness of the text features in the target feature map. At this time, text recognition based on the target feature map can improve the accuracy of text recognition.

[0105] Figure 5 is a schematic block diagram of a text recognition device 500 provided in one embodiment of the present application. It should be understood that Figure 5 The device 500 shown is only an example, and the device 500 of the embodiment of the present application may also include other modules or units. It should be understood that the device 500 is capable of executing Figure 2 and Figure 3 To avoid repetition, the steps in the method are not described in detail here.

[0106] A correction unit 510, configured to correct the direction of text in an input image to obtain a candidate image, wherein the input image is a building image;

[0107] An acquisition unit 520, configured to acquire a feature map of the candidate image;

[0108] A feature enhancement unit 530 is used to perform feature enhancement on the feature map based on the context information of the text to obtain a target feature map;

[0109] The text recognition unit 540 is used to perform text recognition based on the target feature map to obtain a text recognition result of the input image.

[0110] Optionally, the correction unit 510 is specifically used to: perform semantic segmentation on the input image to obtain a semantic segmentation map, and the semantic segmentation map is used to indicate the text in the input image; perform text detection on the input image based on the semantic segmentation map to obtain a text detection result, and the text detection result is used to indicate the text area in the input image; correct the direction of the text based on the text detection result to obtain the candidate image.

[0111] Optionally, the directions of the texts in the candidate images are consistent.

[0112] Optionally, the feature enhancement unit 530 is specifically used to: use a deep learning model to enhance the feature map based on the context information of the text to obtain a target feature map, and the deep learning model includes a bidirectional model structure consisting of an encoder, a decoder and an attention mechanism.

[0113] Optionally, the feature enhancement unit 530 is specifically used to: use a feature pyramid network to convolve the feature map to obtain a multi-scale feature map; use the deep learning model to enhance the multi-scale feature map based on the context information of the text to obtain a target feature map.

[0114] Optionally, the text recognition unit 540 is specifically used to: use a connection temporal classification CTC model to perform text recognition based on the target feature map to obtain the text recognition result.

[0115] Optionally, the acquisition unit 520 is specifically used to: extract a feature map of the candidate image using a lightweight network.

[0116] Optionally, the lightweight network consists of a separation convolution, a bottleneck layer including a 3x3 convolution, and a pooling layer.

[0117] It should be understood that the device 500 here is embodied in the form of a functional module. The term "module" here can be implemented in the form of software and / or hardware, and is not specifically limited to this. For example, a "module" can be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a proprietary processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a merged logic circuit, and / or other suitable components that support the described functions.

[0118] As an example, the device 500 provided in the embodiment of the present application may be a processor or a chip for executing the method described in the embodiment of the present application.

[0119] Figure 6 It is a schematic block diagram of a text recognition device 400 according to an embodiment of the present application. Figure 6 The device 400 shown includes a memory 401, a processor 402, a communication interface 403 and a bus 404. The memory 401, the processor 402 and the communication interface 403 are connected to each other through the bus 404.

[0120] The memory 401 may be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 401 may store a program. When the program stored in the memory 401 is executed by the processor 402, the processor 402 is used to execute each step of the method of the embodiment of the present application. For example, the processor 402 may execute Figure 2 and Figure 3 The various steps of the illustrated embodiment.

[0121] The processor 402 can adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the method of the method embodiment of the present application.

[0122] The processor 402 may also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the method of the embodiment of the present application may be completed by an integrated logic circuit of hardware in the processor 402 or by instructions in the form of software.

[0123] The processor 402 may also be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0124] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 401, and the processor 402 reads the information in the memory 401, and combines its hardware to complete the functions required to be executed by the units included in the text recognition device in the embodiment of the present application, or execute the method of the method embodiment of the present application, for example, it can be executed Figure 2 and Figure 3 The various steps / functions of the illustrated embodiment.

[0125] The communication interface 403 may use, but is not limited to, a transceiver or other transceiver device to implement communication between the apparatus 400 and other devices or a communication network.

[0126] The bus 404 may include a path for transmitting information between various components of the device 400 (eg, the memory 401 , the processor 402 , and the communication interface 403 ).

[0127] It should be understood that the device 400 shown in the embodiment of the present application may be a processor or a chip for executing the method described in the embodiment of the present application.

[0128] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0129] It should be understood that in the embodiment of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0130] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0131] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0136] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for text recognition, characterized in that: include: Correcting the direction of text in an input image to obtain a candidate image, wherein the input image is a building image; Inputting the candidate image into a text recognition model to obtain a text recognition result of the input image; The text recognition model includes a lightweight network, a feature pyramid network, and a bidirectional transformer model; The step of inputting the candidate image into a text recognition model to obtain a text recognition result of the input image includes: Extracting a feature map of the candidate image through the lightweight network; Convolving the feature map through the feature pyramid network to obtain a multi-scale feature map; By using the bidirectional transformer model and based on the context information of the text, the multi-scale feature map is enhanced to obtain a target feature map; Text recognition is performed based on the target feature map to obtain the text recognition result.

2. The method according to claim 1, characterized in that The step of correcting the direction of the text in the input image to obtain a candidate image includes: Performing semantic segmentation on the input image to obtain a semantic segmentation map, wherein the semantic segmentation map is used to indicate text in the input image; Performing text detection on the input image based on the semantic segmentation map to obtain a text detection result, wherein the text detection result is used to indicate a text area in the input image; The direction of the text is corrected based on the text detection result to obtain the candidate image.

3. The method according to claim 2, characterized in that The directions of the texts in the candidate images are consistent.

4. The method according to claim 1, characterized in that The performing text recognition based on the target feature map to obtain a text recognition result of the input image includes: The connection temporal classification (CTC) model is used to perform text recognition based on the target feature map to obtain the text recognition result.

5. The method according to claim 1, characterized in that: The lightweight network consists of a separation convolution, a bottleneck layer including a 3x3 convolution, and a pooling layer.

6. A text recognition device, characterized in that: include: A correction unit, used for correcting the direction of text in an input image to obtain a candidate image, wherein the input image is a building image; A text recognition unit, used for inputting the candidate image into a text recognition model to obtain a text recognition result of the input image; The text recognition model includes a lightweight network, a feature pyramid network, and a bidirectional transformer model; The text recognition unit comprises: An acquisition subunit, configured to extract a feature map of the candidate image through the lightweight network; A multi-scale convolution subunit, used to convolve the feature map through the feature pyramid network to obtain a multi-scale feature map; A feature enhancement subunit, used to enhance the multi-scale feature map through the bidirectional transformer model and based on the context information of the text to obtain a target feature map; The text recognition subunit is used to perform text recognition based on the target feature map to obtain the text recognition result.

7. A device for text recognition, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 5.

Images