Optical character recognition method and apparatus and electronic device

Abstract

An optical character recognition method and apparatus and an electronic device. The optical character recognition method comprises: acquiring an input text image (S301); decoding visual feature vectors obtained by encoding the text image, so as to obtain a first prediction result (S302); and iteratively optimizing the first prediction result to obtain a second prediction result, wherein during the current round of iteration, a query vector matrix and a key vector matrix are each computed by using a position embedding vector during at least one attention computation, and a value vector matrix is computed by using a position embedding vector and a character embedding vector, and when the current round of iteration is the same as the first round of iteration, the character embedding vector is the first prediction result, and when the current round of iteration is different from the first round of iteration, the character embedding vector is a prediction result obtained from the previous round of iteration of the current round of iteration (S303). In this way, during the injection of contextual semantic information, corresponding positional information can be provided by simply using neighboring characters, without relying on excessive contextual semantic information, thereby simplifying an attention mechanism and improving an optimization effect.

Description

Optical character recognition method, device and electronic equipment

[0001] The present application claims priority to the Chinese patent application No. 202410906649.3, filed on July 5, 2024, and entitled "Optical character recognition method, device and electronic equipment", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to an optical character recognition method, device and electronic equipment. BACKGROUND

[0003] Optical character recognition (OCR) technology plays an important role in many fields, such as: certificate recognition, license plate recognition, text recognition in advertisement pictures or analysis of bill information, etc. In the application of OCR technology, there is a high requirement for the accuracy and reliability of the OCR technology. At present, autoregressive decoding mechanism is often used for OCR recognition. This mechanism can utilize the context information of the text to improve the accuracy of recognition. However, this mechanism needs to generate the text character by character in sequence, which limits the possibility of parallel processing, resulting in slow overall processing speed and increasing inference latency. Therefore, how to efficiently, accurately and reliably perform OCR recognition is a technical problem to be solved at present. SUMMARY

[0004] The present application provides an optical character recognition method, device, electronic equipment, computer storage medium and computer product, which can efficiently, accurately and reliably perform OCR recognition.

[0005] In a first aspect, the present application provides an optical character recognition method, comprising: obtaining an input text image; decoding a visual feature vector obtained from the text image to obtain a first prediction result; and iteratively optimizing the first prediction result to obtain a second prediction result, wherein in the current round of iteration, the query vector matrix and the key vector matrix are calculated based on the position embedding vector at least once in the attention calculation process, and the value vector matrix is calculated based on the position embedding vector and the character embedding vector, and when the current round is the same as the first round, the character embedding vector is the first prediction result, and when the current round is different from the first round, the character embedding vector is the prediction result obtained in the last round of iteration of the current round.

[0006] In this way, since the Q matrix and the K matrix are calculated only by using the position embedding vector when attention calculation is performed in the iterative optimization process, the corresponding position information can be provided only by using the adjacent characters in the process of injecting the context semantic information, without relying on too much context semantic information, thereby simplifying the attention mechanism and improving the optimization effect. Meanwhile, since the context semantic information is not relied on too much, the degree of decline in the optimization effect caused by errors in the context semantic information can be reduced.

[0007] In a possible implementation, the method further includes: in the current round iteration process, performing attention calculation by using an attention mask matrix. In this way, which parts in the attention mechanism can "see" each other can be controlled by the attention mask matrix, thereby meeting the requirements of subsequent optimization tasks.

[0008] In a possible implementation, when the current round is an odd round, the attention mask matrix is a preset attention mask matrix; and when the current round is an even round, the attention mask matrix is calculated based on the preset attention mask matrix and the difference between the prediction results obtained in the two rounds adjacent to the current round. In this way, the attention mask matrix can be adaptively adjusted in the iteration process, thereby introducing more correct context semantic information in the iterative optimization process.

[0009] In a possible implementation, before the visual feature vector obtained by encoding the text image is decoded, the method further includes: decomposing the text image into a plurality of non-overlapping two-dimensional small blocks; and performing feature extraction on the two-dimensional blocks to obtain the visual feature vector. In this way, the visual feature vector of the text image can be extracted by image decomposition.

[0010] In a second aspect, the present application provides an optical character recognition device, which includes an acquisition module and a processing module. The acquisition module is configured to acquire an input text image. The processing module is configured to decode a visual feature vector obtained by encoding the text image to obtain a first prediction result. In addition, the processing module is further configured to iteratively optimize the first prediction result to obtain a second prediction result. In the current round iteration process, at least in one attention calculation process, a query vector matrix and a key vector matrix are calculated by using a position embedding vector, and a value vector matrix is calculated by using the position embedding vector and a character embedding vector. When the current round is the same as the first round, the character embedding vector is the first prediction result. When the current round is different from the first round, the character embedding vector is a prediction result obtained in the last iteration of the previous round of the current round.

[0011] In a possible implementation, the processing module is further configured to: in the current round iteration process, perform attention calculation by using an attention mask matrix.

[0012] In a possible implementation, when the current round is an odd round, the attention mask matrix is a preset attention mask matrix; when the current round is an even round, the attention mask matrix is calculated based on the preset attention mask matrix and a difference between prediction results obtained in two rounds adjacent to the current round.

[0013] In a third aspect, the present application provides an electronic device, comprising: at least one memory for storing a program; at least one processor for executing the program stored in the memory; wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is run on a processor, the processor executes the method described in the first aspect.

[0015] In a fifth aspect, the present application provides a computer program product, which, when run on a processor, causes the processor to execute the method described in the first aspect.

[0016] In a sixth aspect, the present application provides an OCR recognition device, comprising at least one processor and an interface; the at least one processor obtains program instructions through the interface; the at least one processor executes the program instructions to implement the method described in the first aspect. Exemplarily, the device can be but is not limited to a chip.

[0017] It can be understood that the beneficial effects of the above-mentioned second aspect to the sixth aspect can be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0018] FIG. 1 is a schematic diagram of an architecture of an OCR recognition system according to an embodiment of the present application;

[0019] FIG. 2 is a similarity visualization diagram of a visual feature vector in autoregressive decoding and parallel decoding, and a similarity visualization diagram of a position embedding vector according to an embodiment of the present application;

[0020] FIG. 3 is a flowchart of an OCR recognition method according to an embodiment of the present application;

[0021] FIG. 4 is a schematic diagram of a structure of an OCR recognition device according to an embodiment of the present application;

[0022] FIG. 5 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application;

[0023] FIG. 6 is a schematic diagram of another structure of an OCR recognition device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The term “and / or” used in the present document is used to describe the association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A existing alone, A and B existing simultaneously, and B existing alone. The symbol “ / ” in the present document represents an or relationship of associated objects, for example, A / B represents A or B.

[0025] The terms “first” and “second” and the like in the description and claims of the present document are used to distinguish different objects, rather than to describe a specific order of the objects. For example, the first response message and the second response message are used to distinguish different response messages, rather than to describe a specific order of the response messages.

[0026] In the embodiments of the present application, the words such as “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design scheme described as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design schemes. Rather, the words such as “exemplary” or “for example” are used in the specific manner to present the relevant concept.

[0027] In the description of the embodiments of the present application, unless otherwise specified, “a plurality of” means two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like.

[0028] Generally, when performing OCR recognition, a parallel decoding technology based on a cross-attention mechanism can be used to overcome the limitations brought by the autoregressive decoding mechanism. The parallel decoding technology can allow the output of the entire sequence to be generated at the same time. Although this can improve the recognition efficiency, the accuracy and reliability are relatively low. In order to improve the accuracy and reliability of recognition when using the parallel decoding technology, an iterative optimization method can be used to correct the recognition result. However, in the iterative optimization process, more context semantic information is needed, and when the context semantic information is incorrect, the optimization effect is not ideal. Compared with the autoregressive decoding mechanism, the OCR recognition mechanism using parallel decoding + iterative optimization can achieve relatively efficient, relatively accurate and relatively reliable OCR recognition, but since more context semantic information is still needed in the iterative optimization process, the accuracy and reliability of recognition are limited.

[0029] In view of this, the embodiment of the present application provides an OCR recognition system, which can utilize the characteristics of visual information in a text image based on a position-guided self-attention mechanism, reduce the dependence on context semantic information in the iterative optimization process, simplify the original attention calculation scheme, and thus realize efficient, accurate and reliable OCR recognition. In addition, the OCR recognition system can also adaptively adjust the attention mask matrix in the iterative optimization process, so that more correct context semantic information can be introduced in the optimization process to achieve better optimization effect. Illustratively, the text image can be an image containing text information. The OCR recognition system is described below.

[0030] Illustratively, FIG. 1 shows an architecture schematic diagram of an OCR recognition system provided by the embodiment of the present application. As shown in FIG. 1, the OCR recognition system 100 can include a visual processing module 110, a parallel decoding module 120, a classification head 130, an iterative optimization module 140 and a classification head 150.

[0031] The visual processing module 110 is mainly used for encoding the collected text image into visual features. The visual processing module 110 can decompose the text image into a plurality of non-overlapping two-dimensional small blocks. Then, the visual processing module 110 can extract features from the decomposed two-dimensional small blocks respectively to obtain a visual feature vector of the text image. Illustratively, the features extracted from any two-dimensional small block can be understood as a token. In some embodiments, the visual processing module 110 is but not limited to a vision transformer (ViT).

[0032] The parallel decoding module 120 is mainly used for parallel decoding the visual feature vector output by the visual processing module 110. The parallel decoding module 120 can include a multi-head attention network (MHA) 121 and a feed-forward network (FFN) 122. The MHA 121 is mainly used for attention calculation on the visual feature vector output by the visual processing module 110. The FFN 122 is mainly used for further nonlinear transformation on the output of the MHA 121 to increase the expression ability and performance of the model.

[0033] In the embodiment, the query vector (Q) matrix, the key vector (K) matrix and the value vector (V) matrix used by the MHA 121 when performing attention calculation can be respectively: Q K = X × W K V = X × W V W Q, W K and W V are learned weight matrices for linear transformation; X is a sequence of visual feature vectors output by the visual processing module 110; A is a position embedding vector, which can provide position information of each position in the input sequence. For example, the calculation formula of the MHA 121 when performing attention calculation can be as follows:

[0034] where d is the dimension of Q and K.

[0035] The classification head 130 is mainly used to classify the output of the parallel decoding module 120 to obtain the prediction result of each character at each time step in the text image. For each time step, the classification head 130 can output a probability distribution representing the probability of each character. Then, by selecting the character with the highest probability, the character prediction result at the corresponding time step is obtained. For example, a time step can represent each position in a small segment or feature sequence divided in the text image. Each time step corresponds to a character position, and the classification head 130 can output the prediction result of the corresponding character at each time step. In this embodiment, the output of the classification head 130 can be transmitted to the iterative optimization module 140 to perform iterative optimization on the prediction result by the iterative optimization module 140.

[0036] The iterative optimization module 140 is mainly used to perform iterative optimization processing on the prediction result output by the classification head 130 to obtain the final prediction result. The iterative optimization module 140 can include MHA 141, MHA 142, and FFN 143. MHA 141 and 142 are used to perform attention calculation, and FFN 143 is used to further nonlinearly transform the output of MHA 142 to increase the expression ability and performance of the model.

[0037] where for MHA 141, the Q matrix, K matrix, and V matrix used for calculation at each iteration can be Q=A×W′ Q , K=A×W′ K , and V=(X′+A)×W′ V respectively. W′ Q , W′ K , and W′ V are learned weight matrices for linear transformation; X′ is an input sequence, which can also be referred to as a “character embedding vector”; A is a position embedding vector, which can provide position information of each position in the input sequence. In the first iteration, X′ is the prediction result output by the classification head 130; in other iterations except the first iteration, X′ is the prediction result output by the classification head 150. In this embodiment, the output of MHA 141 at the nth iteration can be h n= MHA(Pos, Pos, V n , M n , M n = Pos + Embedding(argmax(y n-1 ))

[0038] wherein, Pos is a position embedding vector; Embedding(argmax(y n-1 )) is a prediction result obtained in the n-1th iteration, wherein, when n-1 = 0, Embedding(argmax(y n-1 )) is a prediction result output by the classification head 130; y n-1 is a probability matrix showing the recognition result of the previous round; argmax(y n-1 ) is to select the character with the maximum prediction probability from the character pool as the output; M n is an attention mask matrix (attention_mask) used in the n th iteration.

[0039] In addition, in the n th iteration optimization process, the calculation formula of MHA 141 when performing attention calculation can be as follows:

[0040] wherein, the position embedding vector Pos and are known, so M Pos can be calculated in advance, which reduces the calculation amount of MHA 141, thereby improving the calculation efficiency. At the same time, the context semantic information is not introduced into the Q matrix and the K matrix used by MHA 141, but only the adjacent characters are used to provide the corresponding position information, and only the context semantic information is introduced into the V matrix, so that the dependence on the context semantic information is reduced.

[0041] In addition, in order to be able to introduce more correct context semantic information in the iterative optimization process, the attention mask matrix M n can also be adaptively adjusted. When n is odd, M n is the system default attention mask matrix, and when n is even, M nA new attention mask matrix can be obtained by comparing the latest two prediction results before this round of iteration and the system default attention mask matrix. The new attention mask matrix can mask the characters that cannot be compared in the latest two prediction results before this round of iteration. For example, if the result output by the classification head 130 is [c, i, o, t, h, i, n, p], and the prediction result obtained in the first round of iteration is [c, I, o, t, h, l, n, p], then in the second round of iteration, since the second character and the sixth character in the two prediction results cannot be compared, it indicates that the characters at these two positions may be incorrect, so in this round of optimization process, a new attention mask matrix can be used to mask these two pieces of information to avoid introducing incorrect context information to affect the optimization process. The new attention mask matrix can be obtained by comparing the results of the previous two prediction results and the system default attention mask matrix. The right side of FIG. 1 shows the process of obtaining the new attention mask matrix. In the right side of FIG. 1, in the i-th round of iteration, comparing the previous two prediction results can obtain a mask matrix M c , and combining M c with the system default attention mask matrix M D , the attention mask matrix M i used in the i-th round of iteration can be obtained. In this embodiment, the attention mask matrix used in the n-th round of iteration can be obtained by the following formula: Text n = argmax (y n )

[0042] wherein RM is a component in the iterative optimization module 140 that performs attention mechanism, Text n represents the text output by the n-th round of iteration optimization (n = 0 represents the text output by the classification head 130), M pad represents the system default attention mask matrix.

[0043] MHA 142 is mainly used to perform attention calculation with the output of MHA 141 and the output of the visual processing module 110 as input. The Q matrix, K matrix and V matrix used by MHA 142 for attention calculation can be Q = B x W" Q , K = X x W" K , and V = X x W" V . W" Q , W" K and W" Vare learned weight matrices used for linear transformation; X is the sequence of visual feature vectors output by the visual processing module 110; and B is the output of the MHA 141. Since more accurate context information is introduced in the output of the MHA 141, the output of the MHA 142 will be more accurate and reliable.

[0044] The classification head 150 is mainly used to classify the output obtained by the iterative optimization module 140 in each round to obtain the prediction result of each character in each time step in the text image of each round of iteration. Wherein, when the number of iterations does not reach the preset number or the iteration time does not reach the preset time, the output of the classification head 150 can be passed to the MHA 141 for the next round of iteration.

[0045] In the above-mentioned OCR recognition system 100, by using the position-guided attention mechanism, in the iterative optimization stage, the rich visual information of the OCR model can be used, in the process of injecting context semantic information, only the adjacent characters are used to provide the corresponding position information, without relying on too much context semantic information, thereby simplifying the attention mechanism and improving the optimization effect. In addition, in the iterative optimization process, the adaptive adjustment of the attention mask matrix can also be used to cover the wrong character information in the mask process, so that more correct context information can be provided in the iteration process to obtain better iterative optimization effect. In this way, efficient, accurate and reliable OCR recognition is realized.

[0046] In training the OCR recognition system 100 shown in FIG. 1, as shown in FIG. 2, (A) of FIG. 2 shows a similarity visualization diagram of the position embedding vector in autoregressive decoding, (B) of FIG. 2 shows a similarity visualization diagram of the position embedding vector in parallel decoding, (C) of FIG. 2 shows a similarity visualization diagram of the visual feature vector in autoregressive decoding, and (D) of FIG. 2 shows a similarity visualization diagram of the visual feature vector in parallel decoding. As can be seen from FIG. 2, autoregressive decoding causes the position embedding vector to focus on irrelevant areas, and parallel decoding causes attention to diverge under long text. Considering the influence of the two training strategies on the model, a part of autoregressive decoding is used in the iterative optimization module, and the gradient flow between the parallel decoding module 120 and the iterative optimization module 140 is used to achieve better training effect and improve the expression ability of the overall model. During the training of the OCR recognition system 100, the loss L1 can be obtained by processing the output of the classification head 130 and the sample label through the loss function, and the loss L2 can be obtained by processing the output of the classification head 150 after the iterative optimization and the sample label through the loss function. Then, the parameters in the parallel decoding module 120 and the classification head 130 are adjusted through the loss L1, and the parameters in the iterative optimization module 140 and the classification head 150 are adjusted through the loss L2. At the same time, the parameters in the visual processing module 110 can be adjusted through the losses L1 and L2. In this way, the training of the OCR recognition system 100 can be realized. For example, when optimizing the position embedding vector, only the loss L1 can be used to avoid the risk of focusing on irrelevant areas caused by the loss L2.

[0047] It should be understood that the OCR recognition system 100 shown in FIG. 1 can be configured on any electronic device, such as a mobile phone, a computer, etc. Of course, the OCR recognition system 100 can also be configured on the cloud to realize OCR recognition on the cloud side. In addition, the OCR recognition system 100 can also be partially configured on the terminal side and partially configured on the cloud side to realize OCR recognition through terminal-cloud collaboration. The configuration of the OCR recognition system 100 can be determined according to actual conditions, which is not limited here.

[0048] The above is a related introduction to the OCR recognition system provided by the embodiments of the present application. Based on the above, an OCR recognition method provided by the embodiments of the present application is introduced below.

[0049] Exemplarily, FIG. 3 shows a flowchart of an OCR recognition method provided by an embodiment of the present application. It can be understood that the method can be executed by any device, equipment, platform, or cluster of equipment having computing and processing capabilities. Exemplarily, the method can be executed by an OCR recognition device, which can be implemented by software and / or hardware, and can be configured in an electronic device or a server, typically, in a computer or a mobile phone. For ease of description, the following will take an electronic device as an execution subject for introduction. As shown in FIG. 3, the OCR recognition method can include the following steps:

[0050] S301, obtaining an input text image.

[0051] In this embodiment, the text image can be uploaded by a user to an electronic device, can be photographed by the user using the electronic device, or can be transmitted to the electronic device by other devices or equipment, and the specific manner can be determined according to actual conditions, which is not limited here.

[0052] S302, decoding a visual feature vector obtained by encoding the text image to obtain a first prediction result.

[0053] In this embodiment, after obtaining the text image, the text image can be encoded to obtain a visual feature vector of the text image. Exemplarily, when the text image is encoded, the text image can be first decomposed into a plurality of non-overlapping two-dimensional small blocks; then, feature extraction is performed on each two-dimensional block, for example, the two-dimensional blocks are processed by a convolution layer, to obtain the visual feature vector of the text image. For example, the encoding can be performed by the visual processing module 110 in the OCR recognition system 100 shown in FIG. 1.

[0054] After obtaining the visual feature vector of the text image, the visual feature vector can be decoded to obtain a first prediction result. Exemplarily, when decoding, the visual feature vector can be decoded by a parallel decoding technology to improve decoding efficiency. In the decoding process, when attention calculation is performed, the Q matrix can be calculated by the position embedding vector, and the K matrix and the V matrix can be calculated by the visual feature vector. For example, the decoding can be performed by the parallel decoding module 120 in the OCR recognition system 100 shown in FIG. 1.

[0055] S303, iteratively optimizing the first prediction result to obtain a second prediction result, wherein, in the current round of iteration, the query vector matrix and the key vector matrix are calculated by the position embedding vector at least once in the attention calculation process, and the value vector matrix is calculated by the position embedding vector and the character embedding vector, the character embedding vector being the first prediction result when the current round is the same as the first round, and the character embedding vector being the prediction result obtained by the last iteration of the current round when the current round is different from the first round.

[0056] In the embodiment, after obtaining the first prediction result, the first prediction result can be iteratively optimized to obtain the second prediction result. In the current round of iteration, the Q matrix and the K matrix are calculated based on the position embedding vector in at least one attention calculation process, and the V matrix is calculated based on the position embedding vector and the character embedding vector. For the character embedding vector, when the current round is the same as the first round, the character embedding vector is the first prediction result; when the current round is different from the first round, the character embedding vector is the prediction result obtained in the last iteration of the previous round of the current round. In some embodiments, the attention calculation described in S303 can be understood as the calculation process of the MHA 141 in the OCR recognition system 100 shown in FIG. 1.

[0057] In this way, since only the position embedding vector is used to calculate the Q matrix and the K matrix when performing attention calculation in the iterative optimization process, the corresponding position information can be provided only by the adjacent characters in the process of injecting context semantic information, without relying on excessive context semantic information, thereby simplifying the attention mechanism and improving the optimization effect. At the same time, since it does not rely on excessive context semantic information, the degree of decline in optimization effect caused by errors in context semantic information can be reduced.

[0058] In addition, in S303, in the current round of iteration, attention calculation can be performed using an attention mask matrix to control which parts in the attention mechanism can see each other, thereby meeting the requirements of subsequent optimization tasks. Further, when the current round is an odd round, the attention mask matrix can be a preset attention mask matrix. When the current round is an even round, the attention mask matrix can be calculated based on the preset attention mask matrix and the difference between the prediction results obtained in the two adjacent rounds of the current round. When the current round is an even round, the attention mask matrix used can mask the characters that cannot be compared in the two latest prediction results obtained before the current round of iteration. In this way, the attention mask matrix can be adaptively adjusted in the iteration process, thereby introducing more correct context semantic information in the iterative optimization process. The source of the attention mask matrix used in each round of iteration is described in detail in the foregoing description of the OCR recognition system 100 shown in FIG. 1, and will not be described again.

[0059] It can be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the technical features described in each of the above embodiments or related to the embodiments can be combined according to the actual situation, and the combined scheme is still within the protection scope of the present application.

[0060] Based on the method in the above embodiments, the embodiments of the present application further provide an OCR recognition device.

[0061] For example, FIG. 4 shows a structural schematic diagram of an OCR recognition device provided by the embodiments of the present application. As shown in FIG. 4, the optical character recognition device 400 can include an acquisition module 401 and a processing module 402. The acquisition module 401 is configured to acquire an input text image. The processing module 402 is configured to decode a visual feature vector obtained by encoding the text image to obtain a first prediction result. In addition, the processing module 402 is further configured to iteratively optimize the first prediction result to obtain a second prediction result, wherein in a current round of iteration, the query vector matrix and the key vector matrix are calculated based on the position embedding vector at least once in the attention calculation process, and the value vector matrix is calculated based on the position embedding vector and the character embedding vector. When the current round is the same as the first round, the character embedding vector is the first prediction result. When the current round is different from the first round, the character embedding vector is the prediction result obtained by the last iteration of the current round.

[0062] In some embodiments, the processing module 402 is further configured to perform attention calculation using an attention mask matrix in the current round of iteration.

[0063] In some embodiments, when the current round is an odd round, the attention mask matrix is a preset attention mask matrix; when the current round is an even round, the attention mask matrix is calculated based on the preset attention mask matrix and the difference between the prediction results obtained by the two adjacent rounds of iteration of the current round.

[0064] In some embodiments, before decoding the visual feature vector obtained by encoding the text image, the processing module 402 is further configured to: decompose the text image into a plurality of non-overlapping two-dimensional small blocks; and perform feature extraction on the two-dimensional blocks to obtain the visual feature vector.

[0065] It should be understood that the above device is used to execute the method in the above embodiments, the corresponding program modules in the device have similar implementation principles and technical effects to those described in the above method, and the working process of the device can refer to the corresponding process in the above method, which will not be described here.

[0066] Based on the method in the above embodiments, the embodiments of the present application further provide an electronic device. As shown in FIG. 5, the electronic device 500 includes a bus 502, a processor 504, a memory 506 and a communication interface 508. The processor 504, the memory 506 and the communication interface 508 communicate through the bus 502. It should be understood that the present application does not limit the number of processors and memories in the electronic device 500.

[0067] The bus 502 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is represented in FIG. 5, but it does not mean that there is only one bus or only one type of bus. The bus 504 can include a path for transmitting information between various components (e.g., the memory 506, the processor 504, the communication interface 508) of the electronic device 500.

[0068] The processor 504 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0069] The memory 506 can include a volatile memory (e.g., a random access memory (RAM)) and / or a non-volatile memory (e.g., a read-only memory (ROM), a floppy disk, a hard disk, or a solid state drive (SSD)).

[0070] The memory 506 stores executable program codes, and the processor 504 executes the executable program codes to respectively implement the functions of the aforementioned modules of FIG. 4, thereby implementing all or part of the steps of the methods of the above-described embodiments. That is, the memory 506 stores instructions for executing all or part of the steps of the methods of the above-described embodiments.

[0071] Alternatively, the memory 506 stores executable program codes, and the processor 504 executes the executable program codes to respectively implement the functions of the aforementioned OCR recognition device 400, thereby implementing all or part of the steps of the methods of the above-described embodiments. That is, the memory 506 stores instructions for executing all or part of the steps of the methods of the above-described embodiments.

[0072] The communication interface 503 uses a transceiving module such as, but not limited to, a network interface card, a transceiver, and the like to enable communication between the electronic device 500 and other devices or communication networks.

[0073] Based on the method in the above embodiment, the embodiment of the present application further provides another OCR recognition device. Please refer to FIG. 6, which is a structural schematic diagram of another OCR recognition device provided by the embodiment of the present application. As shown in FIG. 6, the OCR recognition device 600 comprises one or more processors 601 and an interface circuit 602. Optionally, the OCR recognition device 600 can further comprise a bus 603. Wherein:

[0074] The processor 601 can be an integrated circuit chip with processing capability. In the implementation process, the steps of the above method can be completed by the integrated logic circuit or the instruction in the form of software in the processor 601. The processor 601 mentioned above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods and steps in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor.

[0075] The interface circuit 602 can be used for sending or receiving data, instructions or information. The processor 601 can process the data, instructions or other information received by the interface circuit 602, and can send the processed information out through the interface circuit 602.

[0076] Optionally, the OCR recognition device 600 further comprises a memory, which can include a read-only memory and a random access memory, and provides operation instructions and data for the processor. Part of the memory can also include a non-volatile random access memory (NVRAM).

[0077] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which can be stored in an operating system).

[0078] Optionally, the interface circuit 602 can be used to output the execution result of the processor 601.

[0079] It should be noted that the functions of the processor 601 and the interface circuit 602 correspond to each other, which can be realized by hardware design, software design or a combination of hardware and software, and this is not limited here.

[0080] It should be understood that each step of the above method embodiments can be completed by logic circuit in the form of hardware in the processor or instructions in the form of software.

[0081] Based on the method in the above embodiments, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program runs on a processor, the processor executes the method in the above embodiments. Illustratively, the computer readable storage medium can be any available medium that the electronic device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc.

[0082] Based on the method in the above embodiments, an embodiment of the present application provides a computer program product, which, when running on a processor, makes the processor execute the method in the above embodiments.

[0083] It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0084] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0085] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, cloud computing platform, or data center to another website, computer, cloud computing platform, or data center through a wired (such as a coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a cloud computing platform, data center, etc. that includes one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0086] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for convenient differentiation, and do not limit the scope of the embodiments of the present application.

[0087] Finally, it should be noted that: the above examples are used to illustrate the technical solutions of the present application, but not limited to them; although the present application is described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. An optical character recognition method, characterized by, The method comprises: obtaining an input text image; decoding a visual feature vector obtained from the text image to obtain a first prediction result; iteratively optimizing the first prediction result to obtain a second prediction result, wherein, in a current round of iteration, the query vector matrix and the key vector matrix are calculated based on the position embedding vector in at least one attention calculation process, and the value vector matrix is calculated based on the position embedding vector and the character embedding vector, when the current round is the first round, the character embedding vector is the first prediction result, and when the current round is not the first round, the character embedding vector is the prediction result obtained in the last round of iteration.

2. The method of claim 1, wherein, The method further comprises: in the current round of iteration, attention calculation is performed based on an attention mask matrix.

3. The method of claim 2, wherein, when the current round is an odd round, the attention mask matrix is a preset attention mask matrix; when the current round is an even round, the attention mask matrix is calculated based on the preset attention mask matrix and the difference between the prediction results obtained in the two adjacent rounds of iteration.

4. The method according to any of claims 1 to 3, characterized in that, Before the decoding of the visual feature vector obtained from the text image, the method further comprises: decomposing the text image into a plurality of non-overlapping two-dimensional small blocks; performing feature extraction on the two-dimensional blocks to obtain the visual feature vector.

5. An optical character recognition device, characterized by comprises: an obtaining module that obtains an input text image; a processing module that decodes a visual feature vector obtained from the text image to obtain a first prediction result; The processing module is further configured to iteratively optimize the first prediction result to obtain a second prediction result, wherein, in a current round of iteration, the query vector matrix and the key vector matrix are calculated based on the position embedding vector in at least one attention calculation process, and the value vector matrix is calculated based on the position embedding vector and the character embedding vector, when the current round is the first round, the character embedding vector is the first prediction result, and when the current round is not the first round, the character embedding vector is the prediction result obtained in the last round of iteration.

6. The apparatus of claim 1, wherein, The processing module is further configured to: in the current round of iteration, attention calculation is performed based on an attention mask matrix.

7. The apparatus of claim 6, wherein, when the current round is an odd round, the attention mask matrix is a preset attention mask matrix; when the current round is an even round, the attention mask matrix is calculated based on the preset attention mask matrix and the difference between the prediction results obtained in the two adjacent rounds of iteration.

8. An electronic device, comprising: comprises: at least one memory for storing a program; at least one processor for executing the program stored in the memory; When the program stored in the memory is executed, the processor is configured to perform the method of any one of claims 1-4.

9. A computer-readable storage medium storing a computer program, when the computer program is executed on a processor, the processor is configured to perform the method of any one of claims 1-4.

10. A computer program product, characterised in that, When the computer program product is run on a processor, it causes the processor to perform the method of any of claims 1-4.

Images