Text recognition method, end-to-end system and storage medium

Abstract

The invention provides a text recognition method, an end-to-end system and a storage medium. The method is applied to an end-to-end system capable of outputting a target text in a target image based on the target image. The method comprises the following steps: obtaining the target image, wherein the target image comprises the target text; feature extraction is carried out on the target image to obtain an image feature map, segmentation is carried out on the image feature map in the target direction to obtain N feature sub-maps, and N is an integer larger than 1; text probability information is obtained on the basis of the N feature sub-graphs, the target text is determined on the basis of the text probability information, and the text probability information comprises character information obtained through prediction for each feature sub-graph and the probability corresponding to the character information.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a text recognition method, an end-to-end system, and a storage medium. Background Art

[0002] In modern logistics and automated production, the recognition of printed text on various items is a crucial task. For example, in the production process of consumer goods, the accurate and efficient recognition of printed text on product packaging, such as production date, batch number, etc., is directly related to product quality control, inventory management and other aspects. Therefore, how to improve the accuracy and efficiency of text recognition is an urgent problem to be solved.

[0003] The content of the background technology section is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the invention

[0004] This specification provides a text recognition method, an end-to-end system, and a storage medium, which can improve the efficiency and accuracy of text recognition.

[0005] In a first aspect, the present specification provides a text recognition method, which is applied to an end-to-end system capable of outputting a target text in a target image based on a target image, comprising: obtaining the target image, wherein the target image includes the target text; performing feature extraction on the target image to obtain an image feature map, and segmenting the image feature map in a target direction to obtain N feature sub-maps, wherein N is an integer greater than 1; and obtaining text probability information based on the N feature sub-maps, and determining the target text based on the text probability information, wherein the text probability information includes character information predicted for each feature sub-map and the probability corresponding to the character information.

[0006] In some possible implementations, obtaining the target image includes: obtaining an original image, determining a text region in the original image; and cropping the text region in the original image to obtain the target image.

[0007] In some possible implementations, determining the target text based on the text probability information includes: determining a plurality of candidate texts and the probabilities corresponding to each of the plurality of candidate texts based on the text probability information; and determining the target text based on the probability corresponding to each candidate text.

[0008] In some possible implementations, the N feature subgraphs are ordered, the character information predicted based on each feature subgraph includes at least one character, and multiple candidate texts are determined based on the text probability information, including: performing multiple rounds of searches in sequence according to the order of the N feature subgraphs to determine the multiple candidate texts, wherein: in a first round of search, a first candidate character is determined from the at least one first character based on at least one first character predicted by the first feature subgraph and its corresponding probability, and the first candidate character is used as part of the candidate text; in an i-th round of search, multiple combined characters are determined, and a target combined character is determined from the multiple combined characters as part of the candidate text, 1<i≤N, each combined character includes the first candidate character to the i-1th candidate character, and any i-th character predicted by the i-th feature subgraph; and each partial candidate text obtained by the multiple rounds of searches is used as the multiple candidate texts.

[0009] In some possible implementations, determining a first candidate character from at least one first character based on at least one first character predicted by the first feature subgraph and the corresponding probability includes: selecting M first characters ranked top M in probability from the at least one first character, and using the M first characters as the first candidate characters, where M is a preset integer greater than or equal to 1.

[0010] In some possible implementations, determining the target combined character from the multiple combined characters as part of the candidate text includes: determining, for each combined character, the probability of each character in the combined character being predicted simultaneously; and selecting M combined characters ranked top M in probability as the part of the candidate text based on the probabilities corresponding to the multiple combined characters.

[0011] In some possible implementations, the probabilities corresponding to each of the multiple candidate texts are determined based on the text probability information, including: for each candidate text, determining the probability of each character in the candidate text being predicted simultaneously based on the text probability information and determining the probability corresponding to the candidate text based on the character length in the candidate text.

[0012] In some possible implementations, determining the target text based on the probability corresponding to each candidate text includes: selecting a candidate text with the highest probability among the multiple candidate texts as the target text.

[0013] In a second aspect, the present specification also provides an end-to-end system, comprising: at least one storage medium storing at least one instruction set for performing text recognition; and at least one processor communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set when running, and implements the text recognition method as described in the first aspect according to the instructions of the at least one instruction set.

[0014] Other functions of the text recognition method, end-to-end system, and storage medium provided in this specification will be partially listed in the following description. The creative aspects of the text recognition method, end-to-end system, and storage medium provided in this specification can be fully explained by practicing or using the methods, devices, and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 An application scenario diagram of text recognition provided in this specification is shown;

[0017] Figure 2 The hardware structure diagram of the end-to-end system provided in this specification is shown;

[0018] Figure 3 A flowchart of a text recognition method provided in this specification is shown;

[0019] Figure 4 A schematic diagram of segmenting an image feature map according to the present specification is shown;

[0020] Figure 5 A schematic diagram showing text probability information provided according to the present specification; and

[0021] Figure 6 A schematic diagram of executing beam search according to the present specification is shown. DETAILED DESCRIPTION

[0022] The following description provides specific application scenarios and requirements of this specification, with the purpose of enabling those skilled in the art to make and use the contents of this specification. Various local modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but to the widest scope consistent with the claims.

[0023] The terms used herein are only used for the purpose of describing specific example embodiments and are not restrictive. For example, unless the context clearly indicates otherwise, as used herein, the singular forms "a", "an" and "the" may also include plural forms. When used in this specification, the terms "include", "comprise" and / or "contain" mean that the associated integers, steps, operations, elements and / or components exist, but do not exclude the existence of one or more other features, integers, steps, operations, elements, components and / or groups or that other features, integers, steps, operations, elements, components and / or groups may be added in the system / method.

[0024] In view of the following description, these and other features of the present specification, as well as the operation and function of the related elements of the structure, and the economy of the combination and manufacture of the parts can be significantly improved. Reference is made to the accompanying drawings, all of which form a part of this specification. However, it should be clearly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0025] The flowcharts used in this specification illustrate the operations implemented by the system according to some embodiments in this specification. It should be clearly understood that the operations of the flowcharts may not be implemented in sequence. On the contrary, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0026] Combine the following Figure 1 This section introduces the application scenarios of text recognition provided in this manual.

[0027] Figure 1 FIG. 1 shows a schematic diagram of an application scenario of text recognition provided according to an embodiment of this specification. Figure 1 As shown, scene 001 may include: a photographed object 200 , an image generating device 400 , and an end-to-end system 600 .

[0028] The photographed object 200 is, for example, an object on a production line or a logistics line, such as a packaged food box, etc., and this specification does not impose any limitation on this.

[0029] The image generating device 400 can generate an original image. The image generating device 400 is, for example, a camera, etc., and this specification does not limit this.

[0030] The end-to-end system 600 can recognize text information in the original image based on the original image generated by the image generating device 400. For example, if the original image includes text information such as the production date, the image generating device 400 can obtain the original image including the text information and send the original image to the end-to-end system 600, and the end-to-end system 600 can recognize the text information based on the original image.

[0031] In some embodiments, the end-to-end system 600 may perform the text recognition method described in this specification. The end-to-end system 600 may store data and instructions for implementing the text recognition method, and may execute or be used to execute the data and instructions. In some embodiments, the end-to-end system 600 may include a hardware device with a data information processing function and a necessary program required to drive the hardware device to work.

[0032] Exemplarily, the end-to-end system 600 obtains a target image, the target image includes a target text; extracts features from the target image to obtain an image feature map, and divides the image feature map in the target direction to obtain N feature sub-maps, where N is an integer greater than 1; and obtains text probability information based on the N feature sub-maps, and determines the target text based on the text probability information, the text probability information including character information predicted for each feature sub-map and the probability corresponding to the character information. The specific content is described in detail in the method P300 below and will not be repeated here.

[0033] It should be understood that Figure 1 The number of photographed objects 200, image generating devices 400, and end-to-end systems 600 is merely illustrative. Any number of photographed objects 200, image generating devices 400, and end-to-end systems 600 may be provided as required.

[0034] The end-to-end system 600 may correspond to one device or to a device cluster, which is not limited in this specification. When the end-to-end system 600 corresponds to one device, the text recognition method may be executed entirely on the device. When the end-to-end system 600 corresponds to a device cluster, the text recognition method may be executed in coordination on multiple devices corresponding to the device cluster, or the text recognition method may have other execution modes, which is not limited in this specification.

[0035] It should be noted that, when this specification involves examples of application scenarios, the text information contained in the photographed object 200 is used as an example for description. Those skilled in the art should understand that, in addition to the above-mentioned scenarios, the technical solution provided in this specification can also be applied to other scenarios where text information needs to be obtained by image recognition. The specific implementation methods and technical effects of other scenarios are similar to those of the above-mentioned scenarios, and this specification will not elaborate on them.

[0036] Figure 2 A hardware structure diagram of an end-to-end system 600 provided according to an embodiment of this specification is shown.

[0037] like Figure 2As shown, the end-to-end system 600 may include at least one storage medium 630 and at least one processor 620. In some embodiments, the end-to-end system 600 may also include a communication port 650 and an internal communication bus 610. In addition, the end-to-end system 600 may also include an I / O component 660.

[0038] The internal communication bus 610 can connect different system components. For example, the internal communication bus 610 can connect the storage medium 630, the processor 620, the communication port 650 and the I / O component 660.

[0039] I / O component 660 supports input / output between peer-to-peer system 600 and other components.

[0040] The communication port 650 is used for data communication between the end-to-end system 600 and the outside world. For example, the communication port 650 can be used for data communication between the end-to-end system 600 and the network. The communication port 650 can be a wired communication port or a wireless communication port.

[0041] The network is used to provide a medium for communication connection between the end-to-end system 600 and the image generation device 400. The network can facilitate the exchange of information or data. In some embodiments, the network can be any type of wired or wireless network, or a combination thereof. For example, the network can include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or a similar network. In some embodiments, the network can include one or more network access points. For example, the network can include a wired or wireless network access point, such as a base station or an Internet exchange point, through which one or more components of the end-to-end system 600 and the image generation device 400 can be connected to the network to exchange data or information.

[0042] The storage medium 630 may include a data storage device. The data storage device may be a non-temporary storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. The storage medium 630 also includes at least one instruction set stored in the data storage device. The instruction set may include a computer program code, and the computer program code may include a program, a routine, an object, a component, a data structure, a process, a module, etc. for executing the text recognition method provided in this specification.

[0043] The processor 620 may be in communication with the storage medium 630. The processor 620 is used to execute the at least one instruction set. When the end-to-end system 600 is running, the processor 620 reads the at least one instruction set and executes the text recognition method provided in this specification according to the instructions of the at least one instruction set.

[0044] Processor 620 may be in the form of one or more processors. In some embodiments, processor 620 may include one or more hardware processors. For example, a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, etc., or any combination thereof.

[0045] For the purpose of illustration, only one processor 620 is shown in the end-to-end system 600 in the accompanying drawings. However, it should be noted that the end-to-end system 600 in this specification may also include multiple processors. Therefore, the operations and / or method steps disclosed in this specification may be performed by one processor or jointly by multiple processors. For example, if the processor 620 of the end-to-end system 600 performs step A and step B, it should be understood that step A and step B may also be performed jointly or separately by two different processors 620 (for example, the first processor performs step A, the second processor performs step B, or the first and second processors perform steps A and B together).

[0046] Figure 3 A flowchart of a document recognition method P300 provided according to an embodiment of the present specification is shown. As before, the end-to-end system 600 can execute the document recognition method P300. Specifically, the processor in the end-to-end system 600 can read the instruction set stored in its local storage medium, and then execute the text recognition method P300 according to the provisions of the instruction set.

[0047] The text recognition method P300 can be applied to any platform that needs to obtain text information based on target image recognition. Figure 3 As shown, the text recognition method P300 may include the following steps.

[0048] S310: Obtain a target image, where the target image includes a target text.

[0049] In some embodiments, after obtaining the original image from the image generating device 400 , the end-to-end system 600 identifies the text region in the original image and crops the text region in the original image to obtain the target image.

[0050] For example, if the original image contains three text regions, the three text regions are cropped to obtain three target images. During the cropping process, the text region can be expanded outward by a preset distance so that the cropped text region is more complete and the text information of the text region is not omitted. The following text takes a target image corresponding to a text region as an example to introduce the process of text recognition on the target image. The process of text recognition on target images corresponding to other text regions is similar and will not be repeated.

[0051] Exemplarily, after the end-to-end system 600 obtains the original image, it can perform a series of processing operations on the original image. The processing operations include denoising, improving clarity, etc. The image obtained by the end-to-end system 600 through the processing operations is of higher quality, which is conducive to subsequent accurate text recognition. In other words, after the end-to-end system 600 cuts and obtains the target image, it performs the processing operations on the target image to improve the image quality of the target image.

[0052] Exemplarily, the end-to-end system 600 can identify the text region in the original image through the Character Region Awareness for Text Detection (CRAFT) algorithm, so that the text region can be accurately detected and cropped out to improve the subsequent recognition accuracy. Of course, the end-to-end system 600 can also realize the recognition of the text region through other algorithms, which is not limited in this specification.

[0053] S320: extracting features from the target image to obtain an image feature map, and segmenting the image feature map in the target direction to obtain N feature sub-maps, where N is an integer greater than 1.

[0054] Exemplarily, the end-to-end system 600 can extract features from the target image through a residual network (Residual Networks, ResNet) to obtain an image feature map to complete the conversion of image information into serialized image features. The image feature map represents the visual information of different characters in the target image. In addition, the end-to-end system 600 can also extract the image feature map through other algorithms, which is not limited in this specification. After the image feature map is extracted, Figure 4 As shown, the end-to-end system 600 can also vertically split the image feature map through ResNet to obtain N feature sub-maps. The N feature sub-maps can be identified by sequence information to distinguish the position of each feature sub-map in the image feature map, providing a basis for accurate text recognition.

[0055] S330: Obtaining text probability information based on the N feature subgraphs, and determining a target text based on the text probability information, wherein the text probability information includes text information predicted for each feature subgraph and a probability corresponding to the text information.

[0056] Exemplarily, the end-to-end system 600 is integrated with a bidirectional long short-term memory network (Bidirectional LongShort-Term Memory, BiLSTM) or Deep BiLSTM (relative to BiLSTM, multiple LSTM layers are stacked, which can better learn more complex feature representations and improve network performance). After the end-to-end system 600 inputs N feature subgraphs into the BiLSTM, the text probability information can be obtained. BiLSTM can more comprehensively understand the context in the N feature subgraphs, thereby improving the accuracy of text recognition. Of course, the end-to-end system 600 can also obtain the text probability information through other algorithms, and this specification does not limit this.

[0057] For example, the target text contained in the original image is the character "A", and the end-to-end system 600 divides the image feature map into three feature sub-maps (when N=3). After inputting these three feature sub-maps into the BiLSTM, the following can be obtained: Figure 5 The text probability information shown. The text probability information represents: the characters predicted for the first feature subgraph, and the corresponding probability of each character; the characters predicted for the second feature subgraph, and the corresponding probability of each character; the characters predicted for the third feature subgraph, and the corresponding probability of each character. For example, the probability of predicting the character "A" for the first feature subgraph is a1, the probability of predicting the character "B" for the first feature subgraph is b1, and the probability of predicting the character "A" for the second feature subgraph is a2, and so on.

[0058] In some embodiments, after the end-to-end system 600 obtains the text probability information, it determines multiple candidate texts and the probabilities corresponding to each of the multiple candidate texts based on the text probability information; and then determines the target text based on the probability corresponding to each candidate text. For example, the end-to-end system 600 determines multiple candidate texts and the probabilities corresponding to each of the multiple candidate texts based on the text probability information through a beam search algorithm; and then determines the target text based on the probability corresponding to each candidate text. Beam search can preset the beam width, so that it can balance the accuracy and computational cost, which is more flexible.

[0059] Of course, the end-to-end system 600 may also determine the target text based on the text probability information through other algorithms, and this specification does not limit this.

[0060] The process of determining multiple candidate texts by the end-to-end system 600 is described below.

[0061] In some embodiments, the N feature subgraphs are ordered, and the character information predicted by the end-to-end system 600 based on each feature subgraph includes at least one character. The end-to-end system 600 sequentially performs multiple rounds of searches to determine the multiple candidate texts in the order of the N feature subgraphs.

[0062] Exemplarily, in a first round of search, the end-to-end system 600 determines a first candidate character from the at least one first character based on at least one first character predicted by the first feature subgraph and its corresponding probability, and the first candidate character serves as a partial candidate text.

[0063] For example, the end-to-end system 600 selects M first characters ranked top M in probability from the at least one first character, and uses the M first characters as the first candidate characters, where M is a preset integer greater than or equal to 1.

[0064] Specifically, Figure 6 As shown, assuming that the beam width M = 2, the maximum length of the target text is N (for example, the target text contains at most 3 characters). Assume that the output vocabulary of the N feature subgraphs contains only 5 elements: y = (A, B, C, D, E). For the first feature subgraph, the end-to-end system 600 can obtain the probabilities a1, b1, c1, d1 and e1 corresponding to the first characters "A", "B", "C", "D" and "E" respectively according to the formula P(y1 | c), and determine the characters with the top 2 probabilities. Assuming that the top 2 characters are "A" and "C", "A" and "C" are used as the first candidate characters, that is, characters "A" and "C" are used as candidate texts. In the above formula, c represents the output vocabulary, and y1 represents any first character.

[0065] Exemplarily, in the i-th round of search, the end-to-end system 600 determines multiple combined characters, and determines a target combined character from the multiple combined characters as a partial candidate text, 1<i≤N, and each combined character includes the first candidate character to the i-1th candidate character, and any i-th character predicted by the i-th feature subgraph.

[0066] For example, for each combined character, the end-to-end system 600 determines the probability of each character in the combined character being predicted simultaneously, and selects the top M combined characters in probability ranking as the at least one candidate text based on the probabilities corresponding to the multiple combined characters.

[0067] Specifically, Figure 6As shown, for the second feature subgraph, the end-to-end system 600 calculates the probability corresponding to each combined character (such as "AA", "AB", "AC", "AD", "AE", "CA", "CB", "CC", "CD", "CE") based on the following formula.

[0068] P(A,y2|c)=P(A|c)P(y2|A,c)

[0069] P(C,y2|c)=P(C|c)P(y2|C,c)

[0070] Where y2∈y represents any second character. For example, P(A, B | c) is the probability corresponding to the combined character "AB", which means that when the output word list is c, the probability that the first feature subgraph is predicted as the character "A" and the second feature subgraph is predicted as the character "B". The probabilities corresponding to other combined characters are similar.

[0071] Select the top 2 combination characters among the combination characters "AA", "AB", "AC", "AD", "AE", "CA", "CB", "CC", "CD", and "CE". Assuming that the top 2 characters are "AB" and "CE", "B" and "E" are selected as the second candidate characters, and the combination characters "AB" and "CE" are selected as candidate texts.

[0072] For the third feature subgraph, the end-to-end system 600 calculates the probability corresponding to each combined character (such as "ABA", "ABB", "ABC", "ABD", "ABE", "CEA", "CEB", "CEC", "CED", "CEE") based on the following formula.

[0073] P(A,B,y3|c)=P(A,B|c)P(y3|A,B,c)

[0074] P(C,E,y3|c)=P(C,E|c)P(y3|C,E,c)

[0075] Among them, y3∈y represents any third character. For example, P(A, B, D | c) is the probability corresponding to the combined character "ABD", which can represent the probability that the first feature subgraph is predicted as the character "A", the second feature subgraph is predicted as the character "B", and the third feature subgraph is predicted as the character "D" when the output vocabulary is c. The probabilities corresponding to other combined characters are similar.

[0076] Select the top 2 combination characters in probability from the combination characters "ABA", "ABB", "ABC", "ABD", "ABE", "CEA", "CEB", "CEC", "CED", and "CEE". Assuming that the top 2 characters are "ABD" and "CED", "D" is used as the third candidate character, and the combination characters "ABD" and "CED" are used as candidate texts.

[0077] It can be understood that when N is greater than 3, the search process in the fourth round and higher rounds is similar and will not be described in detail.

[0078] After the Nth round of searching, the end-to-end system 600 uses the candidate texts obtained from the first to the Nth round of searching as the multiple candidate texts. For example, the multiple candidate texts include: character "A", character "C" and combination characters "AB", "CE", "ABD" and "CED".

[0079] In some embodiments, for each candidate text, the end-to-end system 600 determines the probability of each character in the candidate text being predicted simultaneously based on the text probability information and the probability corresponding to the candidate text determined by the length of the characters in the candidate text.

[0080] Exemplarily, the end-to-end system 600 may calculate the probability corresponding to each candidate text by the following formula:

[0081]

[0082] Where L is the number of characters in the candidate text, α can be set to 0.75, y1, ..., yL represent the characters included in a candidate text, N represents the number of feature subgraphs obtained by segmentation, and yN represents the character predicted by the Nth feature subgraph. α It is used to penalize long sequences to avoid the situation where the longer the character length of the candidate text is, the lower the probability is, which is not conducive to outputting accurate target text.

[0083] After obtaining the probability corresponding to each candidate text among the multiple candidate texts, the end-to-end system 600 may select the candidate text with the highest corresponding probability as the target text.

[0084] For example, Figure 6 As shown, assuming that the candidate text "ABD" has the highest corresponding probability, "ABD" is used as the target text finally output by the end-to-end system 600.

[0085] The end-to-end system 600 mentioned above can integrate an end-to-end network model. After the end-to-end system 600 obtains the original image (or the target image), the end-to-end network model can be called to recognize text information in the original image (or the target image).

[0086] The algorithms involved in the end-to-end network model include: an algorithm that can identify and crop text areas (such as CRAFT), an algorithm that can extract features from a target image to obtain an image feature map and divide the image feature map in the target direction to obtain N feature sub-maps (such as ResNet), an algorithm that can obtain text probability information based on N feature sub-maps (such as BiLSTM or DeepBiLSTM), and an algorithm that can determine the target text based on text probability information (such as beam search).

[0087] Before the end-to-end system 600 executes method P300, it is necessary to build an end-to-end network model and train and optimize the end-to-end network model.

[0088] Exemplarily, an end-to-end network model is constructed based on CRAFT, ResNet, DeepBiLSTM and beam search, wherein CRAFT, ResNet and DeepBiLSTM can be regarded as the encoding stage of the end-to-end network model, and beam search can be regarded as the decoding stage of the end-to-end network model.

[0089] During the training process, a loss function of the end-to-end network model can be constructed to calculate the gap between the predicted value and the label value. For example, the connectionist temporal classification (CTC) can be used to construct the loss function of the end-to-end network model to calculate the negative log-likelihood of the predicted value and the label value. Through multiple iterations of training, until the training end condition is met, for example, after a certain iteration of training, the gap between the predicted value and the label value calculated based on the loss function is less than a preset threshold, then the training can be stopped. The end-to-end network model after training and update is integrated into the end-to-end system 600. The end-to-end system 600 can directly output the target text based on the original image (or target image) to improve recognition efficiency. Specifically, whether the end-to-end system 600 can directly output the target text based on the original image or directly output the target text based on the target image depends on whether an algorithm similar to CRAFT is involved in the end-to-end network model.

[0090] The text recognition method, end-to-end system, and storage medium proposed in the embodiments of the present application can directly obtain text information from the original image or the target image through the end-to-end system. The process is relatively simple, and the end-to-end network model can be called only once, which reduces the resources required for model calling and improves recognition efficiency. In addition, the beam search can flexibly select the beam width compared to the greedy algorithm and the exhaustive algorithm, so that a trade-off can be made between the recognition accuracy and the computational load, and a higher recognition accuracy can be achieved within limited computing resources. In addition, the text recognition solution provided in the embodiments of this specification no longer splits the text recognition task into multiple subtasks, so that in actual processing, there is no need to call multiple computing models corresponding to multiple subtasks, which can save the resources consumed by the computing process of the above-mentioned multiple computing models.

[0091] In industrial production lines, since the positions of printed texts on various items are often not fixed, traditional text recognition technology usually requires a complex calibration process for the camera when dealing with such problems. For example, by determining the internal parameters of the camera (such as focal length, optical center position, etc.) and external parameters (such as the position and direction of the camera relative to the packaging box), it is ensured that the camera can focus on specific printed content. However, for situations where the position of printed text is variable, the camera calibration process is more cumbersome. The text recognition method provided in this specification is applied to industrial production lines. Due to the robustness of the deep learning solution to the visual information such as the position and brightness of the identified content, it is not necessary to calibrate the image generation device such as the camera, so as to avoid the text recognition process being too cumbersome, and it can also improve the efficiency of text recognition. And because the end-to-end network model is based on deep learning and iterative training, the text recognition solution provided in this specification can also improve the accuracy of text recognition.

[0092] This specification also provides a computer-readable non-transitory storage medium, storing at least one set of executable instructions for text recognition. When the executable instructions are executed by a processor, the executable instructions guide the processor to implement the steps of the text recognition method described in this specification. In some possible implementations, various aspects of this specification can also be implemented in the form of a program product, which includes a program code. When the program product is run on an end-to-end system 600, the program code is used by the end-to-end system 600 to perform the steps of the text recognition method described in this specification. The program product for implementing the above method can use a portable compact disk read-only memory (CD-ROM) to include program code and can be run on the end-to-end system 600. However, the program product of this specification is not limited to this. In this specification, a readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system or used in combination with it. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media include: an electrical connection with one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. Program code for performing the operations of the present specification may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the end-to-end system 600, partially on the end-to-end system 600, as a stand-alone software package, partially on the end-to-end system 600, partially on a remote computing device, or entirely on a remote computing device.

[0093] The “plurality” mentioned in this article, unless otherwise stated, should be understood as two or more.

[0094] It should also be understood that the first, second and various numerical numbers involved in this document are only distinguished for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0095] It should be understood that the term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0096] Those of ordinary skill in the art will appreciate that the units and steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0098] The above is only a specific implementation of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application, which should be included in the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be based on the protection scope of the claims.

Claims

1. A text recognition method, characterized in that: Applied to an end-to-end system capable of outputting target text in a target image based on a target image, the method comprises: Obtaining the target image, wherein the target image includes the target text; Extracting features from the target image to obtain an image feature map, and segmenting the image feature map in the target direction to obtain N feature sub-maps, where N is an integer greater than 1; and Text probability information is obtained based on the N feature subgraphs, and the target text is determined based on the text probability information, wherein the text probability information includes character information predicted for each feature subgraph and a probability corresponding to the character information.

2. The method according to claim 1, characterized in that The obtaining of the target image comprises: Obtaining an original image, and determining a text area in the original image; and The target image is obtained by cropping the text area in the original image.

3. The method according to claim 1, characterized in that The determining the target text based on the text probability information comprises: Determine a plurality of candidate texts and probabilities corresponding to each of the plurality of candidate texts based on the text probability information; and The target text is determined based on the probability corresponding to each candidate text.

4. The method according to claim 3, characterized in that The N feature subgraphs are ordered, the character information predicted based on each feature subgraph includes at least one character, and a plurality of candidate texts are determined based on the text probability information, including: According to the order of the N feature subgraphs, multiple rounds of searches are performed in sequence to determine the multiple candidate texts, wherein: In the first round of search, a first candidate character is determined from the at least one first character predicted by the first feature subgraph and its corresponding probability, and the first candidate character is used as a partial candidate text; In the i-th round of searching, a plurality of combined characters are determined, and a target combined character is determined from the plurality of combined characters as a partial candidate text, 1<i≤N, and each combined character includes a first candidate character to an i-1th candidate character and any i-th character predicted by an i-th feature subgraph; and The parts of candidate texts obtained through the multiple rounds of searching are used as the multiple candidate texts.

5. The method according to claim 4, characterized in that The step of determining a first candidate character from the at least one first character predicted based on the first feature subgraph and the corresponding probability includes: M first characters ranked top M in probability are selected from the at least one first character, and the M first characters are used as the first candidate characters, where M is a preset integer greater than or equal to 1.

6. The method according to claim 5, characterized in that The step of determining a target combined character from the plurality of combined characters as a partial candidate text comprises: For each combining character, Determining the probability that each character in the combined character is predicted simultaneously; and Based on the probabilities corresponding to the multiple combined characters, M combined characters ranked top M in probability are selected as the partial candidate text.

7. The method according to claim 3, characterized in that The determining the probabilities corresponding to the plurality of candidate texts respectively based on the text probability information includes: For each candidate text, the probability of each character in the candidate text being predicted at the same time is determined based on the text probability information and the probability corresponding to the candidate text is determined based on the length of the characters in the candidate text.

8. The method according to claim 3, characterized in that The determining the target text based on the probability corresponding to each candidate text includes: The candidate text with the highest probability among the multiple candidate texts is selected as the target text.

9. An end-to-end system for outputting target text in a target image based on a target image, comprising: at least one storage medium storing at least one instruction set for performing text recognition; as well as At least one processor is communicatively connected to the at least one storage medium, wherein the at least one processor reads the at least one instruction set when running, and executes the method as described in any one of claims 1-8 according to the instructions of the at least one instruction set.

10. A computer-readable non-transitory storage medium, wherein: The computer-readable non-transitory storage medium stores at least one instruction set, and when the at least one instruction set is executed by at least one processor, the method according to any one of claims 1 to 8 is implemented.

Images