An overseas text recognition method and device, electronic equipment and medium

Abstract

This application discloses an overseas text recognition method, apparatus, electronic device, and medium. The method includes: extracting features from a target overseas text image using a target character recognition network to obtain a target character feature sequence that does not contain character semantic features; aggregating these features based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the feature vector of a preset redundant symbol; and matching each character feature vector in the aggregated character feature sequence with candidate feature vectors in a preset feature library to determine the overseas character corresponding to each character feature vector. This scheme extracts and aggregates target character feature sequences that do not contain character semantic features using a target character recognition network, accurately extracting the character feature vectors corresponding to each character. By comparing these vectors with a preset feature library, the corresponding characters are accurately determined, improving the accuracy of overseas text recognition.

Description

Technical Field

[0001] This application relates to the field of text recognition technology, and in particular to a method, apparatus, electronic device and medium for overseas text recognition. Background Technology

[0002] Text recognition algorithms have become increasingly mature, with deep neural network-based algorithms achieving high performance levels. However, they are heavily reliant on training data, requiring a large amount of labeled data for training. In some scenarios involving overseas text recognition, such as overseas license plate recognition, the language and character styles differ significantly from domestic models, necessitating retraining of the text recognition model. Due to privacy concerns, sufficient training data is often unavailable. Furthermore, the complexity of overseas license plate rules, rapid character updates, and the presence of specific characters in different regions contribute to the low accuracy of overseas license plate character recognition.

[0003] Current solutions for this type of problem include character segmentation-based approaches. These involve extracting each character from overseas license plates, identifying their features, and comparing them with a character feature database to determine the category of each character, ultimately obtaining the license plate number. However, segmenting each character increases the risk of algorithm errors, and extracting features from each segmented character separately increases the algorithm's processing time. Summary of the Invention

[0004] This application provides an overseas text recognition method, device, electronic device, and medium that can accurately recognize each character in a text without the need for text segmentation.

[0005] According to one aspect of this application, a method for recognizing overseas text is provided, the method comprising:

[0006] Feature extraction is performed on the target overseas text image using a target character recognition network to obtain a target character feature sequence that does not contain character semantic features;

[0007] For each target feature vector in the target character feature sequence, the target feature vectors are aggregated based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of the preset redundant symbol to obtain an aggregated character feature sequence.

[0008] The feature vectors of each character in the aggregated character feature sequence are matched with the candidate feature vectors in the preset feature library. Based on the matching results, the overseas characters corresponding to each character feature vector are determined, and the target overseas text is obtained.

[0009] According to one aspect of this application, an overseas text recognition device is provided, the device comprising:

[0010] The target character feature sequence determination module is used to extract features from the target overseas text image through the target character recognition network to obtain the target character feature sequence that does not contain character semantic features;

[0011] The aggregated character feature sequence determination module is used to aggregate each target feature vector in the target character feature sequence based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, to obtain an aggregated character feature sequence.

[0012] The matching module is used to match each character feature vector in the aggregated character feature sequence with candidate feature vectors in a preset feature library, and determine the overseas character corresponding to each character feature vector based on the matching result to obtain the target overseas text.

[0013] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory that is communicatively connected to at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the overseas text recognition method of any embodiment of this application.

[0017] According to another aspect of this application, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the overseas text recognition method of any embodiment of this application.

[0018] The technical solution of this application embodiment extracts features from a target overseas text image using a target character recognition network to obtain a target character feature sequence that does not contain character semantic features. For each target feature vector in the target character feature sequence, the target feature vectors are aggregated based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the feature vector of a preset redundant symbol, resulting in an aggregated character feature sequence. Each character feature vector in the aggregated character feature sequence is matched with candidate feature vectors in a preset feature library, and the overseas character corresponding to each character feature vector is determined based on the matching result, thus obtaining the target overseas text. The above solution, by extracting a target character feature sequence through a target character recognition network, does not contain character semantic features, accurately obtaining the independent feature vector of a single character, thereby avoiding interference from character semantic features in subsequent feature comparisons of individual characters. By aggregating the target feature vectors to obtain the aggregated character feature sequence, the target feature vectors of each character can be split and deduplicated, removing redundant information and retaining only the character feature vectors belonging to the character. By matching each character feature vector with candidate feature vectors in a preset feature library, individual comparison of single character features is achieved, improving the accuracy of character recognition and avoiding interference from character semantic features.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an overseas text recognition method provided in this application embodiment;

[0022] Figure 2 This is a schematic diagram illustrating the generation of aggregated character feature sequences provided in an embodiment of this application;

[0023] Figure 3 A flowchart of an overseas text recognition method provided in another embodiment of this application;

[0024] Figure 4 A flowchart illustrating an overseas text recognition method provided in yet another embodiment of this application;

[0025] Figure 5 This is a schematic diagram of a single-character recognition network provided in another embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of an overseas text recognition device provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," "third," "fourth," "actual," "preset," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Figure 1 This is a flowchart illustrating an overseas text recognition method provided in an embodiment of this application. This embodiment is applicable to situations involving the recognition of text overseas. The method can be executed by an overseas text recognition device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0031] S110. The target overseas text image is subjected to feature extraction through the target character recognition network to obtain the target character feature sequence that does not contain character semantic features.

[0032] The target character recognition network can be any network capable of retrieving text from an image, namely an OCR (Optical Character Recognition) network. The target overseas text image can be a text image acquired from overseas, whose semantics and rules conform to overseas text standards. Character semantic features are features containing the semantic relationships between characters in the text. For example, "happy day" means a happy day, so the target feature vector for "happy" also contains features reflecting "day". The target character feature sequence is a series of multiple target feature vectors. Since the target overseas text image generally contains more than one character, feature extraction typically yields a target character feature sequence composed of multiple target feature vectors. In the target character feature sequence, multiple target feature vectors may correspond to a single character in a target overseas text image. The number of target feature vectors in the target character feature sequence is generally fixed, determined by the target character recognition network used, for example, 32.

[0033] For example, in practical applications, the target overseas text image to be recognized can be input into a target character recognition network, which then extracts features from the target overseas text image. However, in general, the features extracted from a target overseas text image containing multiple characters include semantic features of the characters. Since overseas text rules are complex and updated rapidly, the semantic features of the characters extracted by the target character recognition network trained previously may not be accurate for the current target overseas text image, affecting subsequent character recognition. Therefore, the network can be trained to extract feature sequences that do not contain semantic features of the characters, thus retaining the target character feature sequences that objectively reflect the character characteristics for subsequent character recognition.

[0034] Specifically, a single-character recognition network can be pre-trained to extract features from only a single character, without including semantic information. During the training of the target character recognition network, it is trained according to a strategy that aligns features with the single-character recognition network. This allows the trained target character recognition network to extract feature sequences from input text images that do not contain semantic features, thus facilitating improvements in the accuracy of overseas character recognition.

[0035] S120. For each target feature vector in the target character feature sequence, the target feature vectors are aggregated according to the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of the preset redundant symbol to obtain an aggregated character feature sequence.

[0036] The preset redundant symbols can be delimiters. In target character recognition networks, when encoding based on CTC (Connectionist Temporal Classification), delimiters are added between identical adjacent characters. Delimiters may or may not be added between different adjacent characters. Therefore, the target feature vector may also contain target feature vectors with delimiters. The redundant symbol feature vector is the pre-obtained target feature vector of the delimiters. Preset redundant symbols can also be other symbols distinct from characters, determined based on specific application scenarios.

[0037] For example, the target character feature sequence contains multiple target feature vectors. Generally, the number of target feature vectors is greater than the number of characters in the target overseas text image. Multiple target feature vectors may correspond to a single character, and the target feature vectors may include feature vectors reflecting pre-defined redundant symbols. To determine which target feature vectors in the target character feature sequence correspond to a single character, the target feature vectors can be aggregated. This involves deduplicating duplicate target feature vectors belonging to the same character and removing redundant symbol feature vectors, retaining only the unique target feature vectors for each character.

[0038] Specifically, the similarity between adjacent target feature vectors can be used to determine whether they are duplicates. If the similarity is high, it indicates that the adjacent target feature vectors are duplicates, and deduplication can be performed. Based on the description above, when using CTC encoding, if there are identical characters at adjacent consecutive positions, a separator is added between the identical characters. That is, the distribution of target feature vectors for duplicate characters should be "target feature vector of identical character; target feature vector of separator; target feature vector of identical character." In this case, the similarity between adjacent target feature vectors is low. Therefore, if a high similarity is detected between adjacent target feature vectors, it does not belong to the above situation of identical adjacent characters, and it can be determined that it belongs to a target feature vector representation of a duplicate character, requiring deduplication. Whether the similarity is high can be determined by setting a preset similarity threshold for comparison.

[0039] The similarity between the target feature vector and the redundant character feature vector can be used to determine whether the target feature vector is a pre-defined redundant character. If the similarity between the target feature vector and the redundant feature vector is high, it indicates that the target feature vector corresponds to a pre-defined redundant character, such as a separator, and is not a real character. In this case, the target feature vector can be removed to avoid affecting the normal recognition of characters. Similarly, a pre-defined similarity threshold can be set to determine whether the similarity is high.

[0040] For example, such as Figure 2 As shown, the extracted target character feature sequence consists of multiple target feature vectors, including duplicate and redundant target feature vectors. After aggregation, deduplication and redundancy are removed, resulting in a simplified aggregated character feature sequence with reduced dimensionality.

[0041] S130. Match each character feature vector in the aggregated character feature sequence with the candidate feature vectors in the preset feature library, determine the overseas character corresponding to each character feature vector based on the matching result, and obtain the target overseas text.

[0042] The preset feature library consists of feature vectors extracted from individual overseas characters. Feature vectors added to the preset feature library become candidate feature vectors. Features extracted from individual overseas characters do not include semantic features between characters. The preset feature library contains candidate feature vectors and the corresponding overseas characters.

[0043] For example, in the aggregated character feature sequence obtained after the above aggregation process, the remaining target feature vectors are the character feature vectors corresponding to each character in the target overseas text image. Each character feature vector can be matched with candidate feature vectors in a preset feature library to determine the overseas character corresponding to each character feature vector based on the matching results, thus achieving the recognition of a single overseas character. Combining these overseas characters yields the target overseas text. For instance, by traversing each character feature vector in the character feature sequence, the similarity between the character feature vector and candidate feature vectors in the preset feature library can be compared. If a candidate feature vector has a similarity exceeding a preset similarity threshold with the currently traversed character feature vector, then the overseas character corresponding to that candidate feature vector is determined to be the character corresponding to the character feature vector. If no candidate feature vector has a similarity exceeding the preset similarity threshold with the currently traversed character feature vector, then the overseas character corresponding to the candidate feature vector with the highest similarity is taken as the character corresponding to the character feature vector.

[0044] The technical solution of this application embodiment extracts features from a target overseas text image using a target character recognition network to obtain a target character feature sequence that does not contain character semantic features. For each target feature vector in the target character feature sequence, the target feature vectors are aggregated based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the feature vector of a preset redundant symbol, resulting in an aggregated character feature sequence. Each character feature vector in the aggregated character feature sequence is matched with candidate feature vectors in a preset feature library, and the overseas character corresponding to each character feature vector is determined based on the matching result, thus obtaining the target overseas text. This solution can extract a target character feature sequence that does not contain character semantic features through a target character recognition network, accurately obtaining the independent feature vector of a single character, thereby avoiding interference from character semantic features on subsequent feature comparisons of individual characters. By aggregating the target feature vectors to obtain the aggregated character feature sequence, the target feature vectors of each character can be split and deduplicated, removing redundant information and retaining only the character feature vectors belonging to the character. By matching each character feature vector with candidate feature vectors in a preset feature library, individual comparison of single character features is achieved, improving the accuracy of character recognition and avoiding interference from character semantic features.

[0045] Figure 3 This is a flowchart illustrating an overseas text recognition method according to another embodiment of this application. This embodiment is an optimization based on the above embodiment; schemes not described in detail in this embodiment are found in the above embodiment. Figure 3 As shown, the method in this embodiment of the application specifically includes the following steps:

[0046] S210. The target overseas text image is subjected to feature extraction through the target character recognition network to obtain the target character feature sequence that does not contain character semantic features.

[0047] S220. For each target feature vector in the target character feature sequence, based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, the type of the target feature vector is determined by comparing the results with a preset similarity threshold. The types include repeated target feature vectors of the same character, redundant symbol feature vectors, and normal characters other than repeated target feature vectors of the same character or redundant symbol feature vectors.

[0048] The preset similarity threshold can be determined based on actual conditions and is used to determine whether two feature vectors are the same vector; for example, it can be set to 0.9. The similarity between two adjacent target feature vectors, and the similarity between a target feature vector and a redundant symbol feature vector, are calculated and compared with the preset similarity threshold to determine whether the target feature vector is a repeated target feature vector of the same character or a redundant feature vector. If the target feature vector is neither a repeated target feature vector of the same character nor a redundant feature vector, it is a normal character feature vector.

[0049] S230. If the target feature vector is a repeated target feature vector of the same character, then only one target feature vector is retained as the character feature vector, and the other repeated target feature vectors are set to empty.

[0050] For example, if consecutive target feature vectors are repeated target feature vectors of the same character, then only one target feature vector needs to be retained as the character feature vector. The other repeated target feature vectors have no effect on subsequent character recognition and can be set to None.

[0051] S240. If the target feature vector is a redundant symbol feature vector, then the target feature vector is set to empty.

[0052] For example, if the target feature vector is a redundant symbol feature vector, that is, it is not the feature vector corresponding to the overseas characters in the target overseas text image, but the feature vector corresponding to the preset redundant symbols, it is useless for character recognition. Therefore, the target feature vector can be set to blank line None.

[0053] S250. If the target feature vector is a normal character, then the target feature vector is retained as a character feature vector.

[0054] If the target feature vector is a normal character, the target feature vector is retained as a character feature, so that overseas characters can be identified by comparing the character feature vector with the candidate feature vectors in the preset feature library.

[0055] After simplification using the S230-S250 scheme, the character feature vectors can be made to correspond one-to-one. It can be clearly determined that the k-th character feature vector corresponds to the k-th overseas character in the target overseas text image, which facilitates accurate and rapid location and recognition of overseas characters.

[0056] In this embodiment of the application, the target feature vectors are aggregated based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, including:

[0057] For the first target feature vector in the target character feature sequence, if the similarity between the first target feature vector and the redundant symbol feature vector is greater than or equal to a preset similarity threshold, then the first target feature vector is determined to be a redundant symbol feature vector.

[0058] Otherwise, determine that the first target feature vector is a normal character;

[0059] Starting from the second target feature vector, traverse each target feature vector in the target character feature sequence. During the traversal, calculate the first similarity between the target feature vector and the previous target feature vector, and the second similarity between the target feature vector and the redundant symbol feature vector.

[0060] If both the first similarity and the second similarity are lower than a preset similarity threshold, then the character corresponding to the target feature vector is determined to be a normal character.

[0061] If the first similarity is greater than or equal to a preset similarity threshold, then the target feature vector is determined to be a repeating target feature vector with the same character;

[0062] If the second similarity is greater than or equal to the preset similarity threshold, then the target feature vector is determined to be a redundant symbol feature vector.

[0063] For example, the specific process of aggregation for target feature vectors in a target character feature sequence can be as follows: For the first target feature vector, calculate the similarity between the first target feature vector and the redundant feature vector. If the similarity is greater than or equal to a preset similarity threshold, then the first target feature vector is determined to be a redundant symbol feature vector, and processed according to the scheme in S230, that is, the vector at that position after aggregation is set to empty. If the similarity is less than the preset similarity threshold, then the first target feature vector is determined to be a normal character, and processed according to the scheme in S250, that is, the target feature vector is retained as the character feature vector at that position after aggregation.

[0064] Starting from the second target feature vector, the process iterates through the vectors. During this process, the first similarity between the target feature vector and the previous target feature vector, and the second similarity between the target feature vector and the redundant symbol feature vector, are calculated. If the first similarity is greater than or equal to a preset similarity threshold, the target feature vector is determined to be a repeating target feature vector of the same character as the previous target feature vector. Since the first target feature vector in a series of consecutively similar target feature vectors is dissimilar to the previous target feature vector, and the first target feature vector has already been retained as a character feature vector, the adjacent similar target feature vectors at this position after aggregation can be set to empty. If the second similarity is greater than or equal to the preset similarity threshold, the target feature vector is determined to be a redundant symbol feature, and is processed according to the scheme in S240. If both the first and second similarities are less than the preset similarity threshold, it means that the target feature vector is neither a repeating target feature vector of the same character nor a preset redundant symbol, and the scheme in S250 is executed.

[0065] Assuming the similarity between feature vectors is represented by the distance between them, the above scheme can be expressed by the formula:

[0066]

[0067] Where z0 represents the first character feature vector after aggregation, z k+1 This represents the (k+1)th eigenvector after aggregation. For z... k When the value is None, the k-th feature vector after aggregation is empty and is automatically skipped. x0 represents the first target feature vector in the target character feature sequence, x k+1 x represents the (k+1)th target feature vector in the target character feature sequence. ∈ It is the feature vector of redundant symbols represented by the separator ∈, θ q This is the feature distance threshold, which is negatively correlated with the preset similarity threshold. It should be noted that the aggregated target feature vectors z... k This is a newly formed series of target feature vectors, which are not directly operated on by the target feature vectors before aggregation. The target feature vectors x before aggregation k The conditional judgments used in the above formulas are still retained.

[0068] In this embodiment, a backward comparison can also be performed, starting from the first target feature vector and traversing it to calculate a first similarity with the next target feature vector. If the first similarity is greater than or equal to a preset similarity threshold, then in the aggregated target feature vector sequence, the previous target feature vector is set to empty, and the next target feature vector is retained as a character feature vector.

[0069] In this embodiment of the application, the method further includes:

[0070] During the traversal, if the first similarity between consecutive target feature vectors and the previous target feature vector is greater than or equal to the preset similarity threshold, then the number of consecutive target feature vectors is counted.

[0071] If the quantity is higher than the preset quantity, an adjustment coefficient is determined based on the quantity and the preset quantity.

[0072] Based on the adjustment coefficient, the preset similarity threshold is increased.

[0073] For example, during the traversal process, if there are consecutive target feature vectors that all satisfy a first similarity score greater than or equal to a preset similarity threshold with the previous target feature vector, then as the number of consecutive target feature vectors increases, the probability that the consecutive target feature vectors are actually repeated target feature vectors of the same character decreases. The reason is that, generally, assuming there are M target features and m overseas characters in the target overseas text image, the number of repeated target feature vectors corresponding to the same character can be approximately calculated to be M / m. Therefore, a preset number can be determined based on this value. If the number of consecutive target feature vectors satisfying the similarity condition exceeds the preset number, the probability that the consecutive target feature vectors are actually repeated target feature vectors of the same character decreases. In this case, the preset similarity threshold can be adaptively increased to reduce the number of target feature vectors misjudged as consecutively similar, thereby improving the accuracy of subsequent aggregation. Specifically, an adjustment coefficient can be determined based on the number of consecutive target feature vectors and the preset number, and this adjustment coefficient can be added to the initial preset similarity threshold to increase the preset similarity threshold. Specifically, the difference between the number of consecutive target feature vectors and a preset number can be calculated. If the difference is positive, the adjustment coefficient can be positively correlated with the difference. If the difference is negative, it is counted as 0.

[0074] Specifically, assuming the similarity between feature vectors is represented by the distance between feature vectors, and the preset similarity threshold is represented by the feature distance threshold, then the above scheme can be expressed by the formula:

[0075] θ q =θ + max(qt,0) × δ;

[0076] θ q θ represents the adjusted feature distance threshold, q represents the original feature distance threshold, t represents the number of consecutive target feature vectors, and max represents the preset number.

[0077] When a target feature vector with a first similarity less than the preset similarity threshold is encountered again during the traversal, the preset similarity threshold is restored to its initial value. That is, q is set to 0 in the above formula.

[0078] S260. Match each character feature vector in the aggregated character feature sequence with the candidate feature vectors in the preset feature library, determine the overseas character corresponding to each character feature vector based on the matching result, and obtain the target overseas text.

[0079] This application provides an overseas text recognition method. Based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and a preset redundant symbol feature vector, the method compares the results with a preset similarity threshold to determine the type of the target feature vector. The types include repeated target feature vectors of the same character, redundant symbol feature vectors, and normal characters other than repeated target feature vectors of the same character or redundant symbol feature vectors. If the target feature vector is a repeated target feature vector of the same character, only one target feature vector is retained as a character feature vector, and the other repeated target feature vectors are set to empty. If the target feature vector is a redundant symbol feature vector, the target feature vector is set to empty. If the target feature vector is a normal character, the target feature vector is retained as a character feature vector. This scheme can directly aggregate based on the similarity between target feature vectors, without relying on the character classification corresponding to each target feature vector. This solves the problem of inaccurate character classification in target character recognition networks affecting the aggregation results, thus achieving accurate aggregation.

[0080] Figure 4 This is a flowchart illustrating an overseas text recognition method according to another embodiment of this application. This embodiment is an optimization based on the above embodiments; schemes not described in detail in this embodiment are found in the above embodiments. Figure 4 As shown, the method in this embodiment of the application specifically includes the following steps:

[0081] S310. Construct a sample set based on a single character image, and train a single character recognition network that uses the same backbone network as the target character recognition network.

[0082] For example, a single-character recognition network can be constructed using the same backbone network as the target character recognition network. A sample set is built based on a single character image, and the single-character recognition network is trained based on the sample set, enabling the single-character recognition network to recognize the feature vector of a single character from a single character image. The single character image used to construct the sample set can be a domestic character image or an overseas character image.

[0083] S320. Input the dataset constructed from individual character images into the single character recognition network to obtain the single character feature vector corresponding to each single character, and determine the single character center feature vector corresponding to the single character based on the single character feature vector corresponding to the same character.

[0084] like Figure 5 As shown, after training the single-character recognition network on a sample set, a dataset is constructed based on single-character images. The single-character recognition network then extracts features to obtain the single-character feature vectors corresponding to each character. If multiple single-character feature vectors exist for the same character, the single-character center feature vector is determined based on these vectors. If only one single-character feature vector exists for the same character, it is directly used as the single-character center feature vector. Since the single-character recognition network is trained on single-character images, the output single-character feature vectors do not contain semantic features. This process yields the single-character center feature vector for each character and also provides knowledge of the corresponding individual characters.

[0085] S330. Input the sample text image into the target character recognition network for training. During the training process of the target character recognition network, make the feature vectors of each sample character in the sample character feature sequence corresponding to the sample text image tend to be consistent with the single character center feature vector of the same character, so that the features extracted by the trained target character recognition network do not contain character semantic features.

[0086] In this embodiment, the sample text images are a large number of easily accessible text images, including both domestic and overseas images. Each sample text image may contain more than one character. The sample text images are input into a target character recognition network to obtain the sample character feature sequence corresponding to the sample text image. This sample character feature sequence also includes character semantic features.

[0087] For example, to remove semantic features from sample character feature sequences during feature extraction by a target character recognition network, the sample character feature vectors obtained in the preceding stages of the network can be aligned with the single-character center feature vectors of each individual character obtained in S320. This ensures that the sample character feature vectors in the final output sequence of the target character recognition network tend to be consistent with the single-character center feature vector of the same character, thus preventing the semantic features extracted by the trained target character recognition network from being included. Specifically, for sample text images, the categories of the characters they contain are known. After directly extracting sample character feature vectors from the sample text image in the target character recognition network, the category of the character corresponding to each sample character feature vector can be determined. The single-character center feature vector of the individual character of that category is found from the results obtained in S320, and the sample character feature vectors tend to be consistent with this single-character center feature vector, so that the semantic features extracted by the trained target character recognition network are not included.

[0088] In this embodiment, the features of each sample character are made to be consistent with the single-character center feature vector of the same character, so that the features extracted by the trained target character recognition network do not contain character semantic features, including:

[0089] For each sample character feature vector in the sample character feature sequence, a feature alignment loss function is constructed based on the sample character feature vector corresponding to the same character and the single character center feature vector;

[0090] The target character recognition network is optimized and adjusted based on the feature alignment loss function and the initialization loss function.

[0091] For example, suppose that for the i-th sample text image, the sample character feature sequence obtained is x yik is the feature vector of the k-th sample character, and yik refers to the character category corresponding to the feature vector of the k-th sample character in the i-th sample text image. yik Let m be the single-character center vector of the overseas characters in the yik category. m is the total number of sample text images, and n is the total number of sample character feature vectors. Then, the feature alignment loss function...

[0092]

[0093] Assume the initial loss function of the target character recognition network is L. CTC Then the total loss function is L = L CTC +L C The target character recognition network is optimized and adjusted based on the overall loss function so that the features extracted by the target character recognition network remove the semantic features of the characters.

[0094] S340. The target overseas text image is extracted using a target character recognition network to obtain a target character feature sequence that does not contain character semantic features.

[0095] S350. For each target feature vector in the target character feature sequence, the target feature vectors are aggregated according to the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of the preset redundant symbol to obtain an aggregated character feature sequence.

[0096] S360. Match each character feature vector in the aggregated character feature sequence with the candidate feature vectors in the preset feature library, determine the overseas character corresponding to each character feature vector based on the matching result, and obtain the target overseas text.

[0097] This application provides an overseas text recognition method. A sample set is constructed based on individual character images, and a single-character recognition network with the same backbone network as the target character recognition network is trained. The dataset constructed from individual character images is input into the single-character recognition network to obtain single-character feature vectors corresponding to each single character. A single-character center feature vector is determined based on the single-character feature vectors corresponding to the same character. Sample text images are input into the target character recognition network to obtain sample character feature sequences corresponding to the sample text images. For each sample character feature vector in the sample character feature sequence, each sample character feature is made to tend to be consistent with the single-character center feature vector of the same character, so that the features extracted by the trained target character recognition network do not contain character semantic features. The above-mentioned scheme can generate single-character feature vectors based on a single-character recognition network. During the training process of the target character recognition network, the sample character feature vectors output by the target character recognition network are aligned based on the single-character feature vectors. This ensures that the features extracted by the trained target character recognition network do not contain character semantic features, thereby improving the accuracy of the feature vectors of individual characters. In addition, compared with the existing technology that segments the text image to be recognized and then recognizes the characters, the scheme in this application first obtains the target character feature sequence output by the target character recognition network that does not contain character semantic features, and then aggregates and uses feature matching to recognize overseas text. This method has a small computational load and high efficiency.

[0098] As a non-limiting implementation, the construction process of the preset feature library includes:

[0099] The overseas character image is input into the single character recognition network to obtain the single character feature vector corresponding to each overseas character, and / or the overseas text image is input into the target character recognition network for feature extraction to obtain the overseas character feature sequence that does not contain character semantic features. The overseas character feature sequence is aggregated and then split to obtain the single character feature vector corresponding to each overseas character.

[0100] For each single character feature vector of the same character, the mean of each single character feature vector, or the cluster center feature vector after clustering, is used as the single character center feature vector of the overseas character.

[0101] The single-character center feature vector is stored in a preset feature library as a candidate feature vector.

[0102] For example, overseas character images can be input into the single-character recognition network trained in the above embodiments to obtain single-character feature vectors corresponding to each overseas character. Overseas single-character images can be obtained by segmenting and cropping overseas text images. Alternatively, each overseas text image can be input into a target character recognition network for feature extraction to obtain an overseas character feature sequence that does not contain character semantic features. This overseas character feature sequence can then be aggregated according to the aggregation method described in the above embodiments to obtain single-character feature vectors containing only the features corresponding to each character. Since there may be more than one single-character feature vector for the same character, the mean of each single-character feature vector can be used, or clustering can be performed. The cluster center feature vectors after clustering can be used as the single-character center feature vectors of the overseas characters. The single-character center feature vectors are stored in a preset feature library as candidate feature vectors. The above-mentioned scheme can obtain the single-character center feature vectors of each individual overseas character to form a preset feature library. Therefore, when performing character recognition, it can specifically compare the feature vectors of each character in the aggregated character feature sequence with each candidate feature vector, and recognize each character one by one. This solves the problem of needing to collect a large amount of data and retrain the model when facing new country characters, as well as the problem of introducing segmentation error in the scheme of character segmentation and recognition of target overseas text images. Thus, without the need for segmentation, it can achieve accurate recognition of overseas characters by performing feature matching based on feature vectors that do not contain character semantic features.

[0103] Figure 6 This is a schematic diagram of an overseas text recognition device provided in an embodiment of this application. This device can execute the overseas text recognition method provided in any embodiment of this application, and possesses the corresponding functional modules and beneficial effects for executing the method. For example... Figure 6 As shown, the device includes:

[0104] The target character feature sequence determination module 410 is used to extract features from the target overseas text image through the target character recognition network to obtain a target character feature sequence that does not contain character semantic features.

[0105] The aggregated character feature sequence determination module 420 is used to aggregate each target feature vector in the target character feature sequence based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, to obtain an aggregated character feature sequence.

[0106] The matching module 430 is used to match each character feature vector in the aggregated character feature sequence with the candidate feature vectors in the preset feature library, and determine the overseas character corresponding to each character feature vector according to the matching result to obtain the target overseas text.

[0107] In this embodiment, the aggregated character feature sequence determination module 420 aggregates the target feature vectors based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, including:

[0108] Based on the similarity between adjacent target feature vectors, and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, the type of the target feature vector is determined by comparing the results with a preset similarity threshold. The type includes repeated target feature vectors with the same character, redundant symbol feature vectors, and normal characters other than repeated target feature vectors or redundant symbol feature vectors with the same character.

[0109] If the target feature vector is a repeated target feature vector of the same character, then only one target feature vector is retained as the character feature vector, and the other repeated target feature vectors are set to empty;

[0110] If the target feature vector is a redundant symbol feature vector, then the target feature vector is set to empty;

[0111] If the target feature vector is a normal character, then the target feature vector is retained as a character feature vector.

[0112] In this embodiment, the aggregated character feature sequence determination module 420 aggregates the target feature vectors based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, including:

[0113] For the first target feature vector in the target character feature sequence, if the similarity between the first target feature vector and the redundant symbol feature vector is greater than or equal to a preset similarity threshold, then the first target feature vector is determined to be a redundant symbol feature vector.

[0114] Otherwise, determine that the first target feature vector is a normal character;

[0115] Starting from the second target feature vector, traverse each target feature vector in the target character feature sequence. During the traversal, calculate the first similarity between the target feature vector and the previous target feature vector, and the second similarity between the target feature vector and the redundant symbol feature vector.

[0116] If both the first similarity and the second similarity are lower than a preset similarity threshold, then the character corresponding to the target feature vector is determined to be a normal character.

[0117] If the first similarity is greater than or equal to the preset similarity threshold, then the target feature vector is determined to be a repeating target feature vector with the same character, and the target feature vector is set to empty;

[0118] If the second similarity is greater than or equal to the preset similarity threshold, then the target feature vector is determined to be a redundant symbol feature vector.

[0119] In this embodiment of the application, the device further includes:

[0120] The quantity statistics module is used to count the number of consecutive target feature vectors if, during the traversal process, the first similarity between consecutive target feature vectors and the previous target feature vector is greater than or equal to a preset similarity threshold.

[0121] The adjustment coefficient determination module is used to determine the adjustment coefficient based on the quantity and the number of intervals between the currently traversed target feature vector and the first target feature vector in a series of consecutive target feature vectors if the quantity is higher than a preset quantity.

[0122] A preset similarity threshold adjustment module is used to increase the preset similarity threshold based on the adjustment coefficient.

[0123] In this embodiment of the application, the device further includes:

[0124] The single-character recognition network training module is used to construct a sample set based on a single character image and train a single-character recognition network that uses the same backbone network as the target character recognition network.

[0125] The single-character center feature vector determination module is used to input the dataset constructed from single character images into the single-character recognition network, obtain the single-character feature vector corresponding to each single character, and determine the single-character center feature vector corresponding to the single character based on the single-character feature vector corresponding to the same character.

[0126] The target character recognition network training module is used to input sample text images into the target character recognition network. During the training process of the target character recognition network, the feature vectors of each sample character in the sample character feature sequence corresponding to the sample text image are made to be consistent with the single character center feature vector of the same character, so that the features extracted by the trained target character recognition network do not contain character semantic features.

[0127] In this embodiment, the target character recognition network training module aims to make the features of each sample character tend to be consistent with the single-character center feature vector of the same character, so that the features extracted by the trained target character recognition network do not contain character semantic features, including:

[0128] For each sample character feature vector in the sample character feature sequence, a feature alignment loss function is constructed based on the sample character feature vector corresponding to the same character and the single character center feature vector;

[0129] The target character recognition network is optimized and adjusted based on the feature alignment loss function and the initialization loss function.

[0130] In this embodiment of the application, the device further includes:

[0131] The single-character feature vector determination module is used to input overseas character images into the single-character recognition network to obtain single-character feature vectors corresponding to each overseas character, and / or input overseas text images into the target character recognition network for feature extraction to obtain overseas character feature sequences that do not contain character semantic features, and then aggregate and split the overseas character feature sequences to obtain single-character feature vectors corresponding to each overseas character.

[0132] The central feature vector determination module is used to determine the mean of the individual character feature vectors of the same character, or the cluster center feature vector after clustering, as the central feature vector of the overseas character.

[0133] The preset feature library construction module is used to store the central feature vector into the preset feature library as a candidate feature vector.

[0134] The overseas text recognition device provided in this application embodiment can execute an overseas text recognition method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.

[0135] Figure 7 A schematic diagram of an electronic device 10, which can be used to implement embodiments of this application, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0136] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless overseas text recognition transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0138] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as overseas text recognition methods.

[0139] In some embodiments, the overseas text recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the overseas text recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the overseas text recognition method by any other suitable means (e.g., by means of firmware).

[0140] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable text recognition device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0144] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data overseas text recognition (e.g., overseas text recognition networks) of any form or medium. Examples of overseas text recognition networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0145] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via overseas text recognition networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0146] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired information of the technical solution of this application can be achieved, and this is not limited herein.

[0147] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for recognizing overseas text, characterized in that, The method includes: Feature extraction is performed on the target overseas text image using a target character recognition network to obtain a target character feature sequence that does not contain character semantic features; For each target feature vector in the target character feature sequence, the target feature vectors are aggregated based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of the preset redundant symbol to obtain an aggregated character feature sequence. The feature vectors of each character in the aggregated character feature sequence are matched with the candidate feature vectors in the preset feature library. Based on the matching results, the overseas characters corresponding to each character feature vector are determined, and the target overseas text is obtained.

2. The method according to claim 1, characterized in that, Based on the similarity between adjacent target feature vectors, and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, the target feature vectors are aggregated, including: Based on the similarity between adjacent target feature vectors, and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, the type of the target feature vector is determined by comparing the results with a preset similarity threshold. The type includes repeated target feature vectors with the same character, redundant symbol feature vectors, and normal characters other than repeated target feature vectors or redundant symbol feature vectors with the same character. If the target feature vector is a repeated target feature vector of the same character, then only one target feature vector is retained as the character feature vector, and the other repeated target feature vectors are set to empty; If the target feature vector is a redundant symbol feature vector, then the target feature vector is set to empty; If the target feature vector is a normal character, then the target feature vector is retained as a character feature vector.

3. The method according to claim 2, characterized in that, Based on the similarity between adjacent target feature vectors, and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, the target feature vectors are aggregated, including: For the first target feature vector in the target character feature sequence, if the similarity between the first target feature vector and the redundant symbol feature vector is greater than or equal to a preset similarity threshold, then the first target feature vector is determined to be a redundant symbol feature vector. Otherwise, determine that the first target feature vector is a normal character; Starting from the second target feature vector, traverse each target feature vector in the target character feature sequence. During the traversal, calculate the first similarity between the target feature vector and the previous target feature vector, and the second similarity between the target feature vector and the redundant symbol feature vector. If both the first similarity and the second similarity are lower than a preset similarity threshold, then the character corresponding to the target feature vector is determined to be a normal character. If the first similarity is greater than or equal to a preset similarity threshold, then the target feature vector is determined to be a repeating target feature vector with the same character; If the second similarity is greater than or equal to the preset similarity threshold, then the target feature vector is determined to be a redundant symbol feature vector.

4. The method according to claim 3, characterized in that, The method further includes: During the traversal, if the first similarity between consecutive target feature vectors and the previous target feature vector is greater than or equal to the preset similarity threshold, then the number of consecutive target feature vectors is counted. If the quantity is higher than the preset quantity, then the adjustment coefficient is determined based on the quantity and the number of intervals between the currently traversed target feature vector and the first target feature vector in the consecutive target feature vectors; Based on the adjustment coefficient, the preset similarity threshold is increased.

5. The method according to claim 1, characterized in that, The process of establishing the target character recognition network includes: A sample set is constructed based on a single character image, and a single character recognition network with the same backbone network as the target character recognition network is trained. The dataset constructed from individual character images is input into a single character recognition network to obtain the single character feature vector corresponding to each single character, and the single character center feature vector corresponding to the single character is determined based on the single character feature vector corresponding to the same character. The sample text image is input into the target character recognition network for training. During the training process of the target character recognition network, the feature vectors of each sample character in the sample character feature sequence corresponding to the sample text image are made to be consistent with the single character center feature vector of the same character, so that the features extracted by the trained target character recognition network do not contain character semantic features.

6. The method according to claim 5, characterized in that, The goal is to make the features of each sample character tend to be consistent with the single-character center feature vector of the same character, so that the features extracted by the trained target character recognition network do not include character semantic features, including: For each sample character feature vector in the sample character feature sequence, a feature alignment loss function is constructed based on the sample character feature vector corresponding to the same character and the single character center feature vector; The target character recognition network is optimized and adjusted based on the feature alignment loss function and the initialization loss function.

7. The method according to claim 5, characterized in that, The construction process of the preset feature library includes: The overseas character image is input into the single character recognition network to obtain the single character feature vector corresponding to each overseas character, and / or the overseas text image is input into the target character recognition network for feature extraction to obtain the overseas character feature sequence that does not contain character semantic features. The overseas character feature sequence is aggregated and then split to obtain the single character feature vector corresponding to each overseas character. For each single-character feature vector of the same character, the mean of each single-character feature vector, or the cluster center feature vector after clustering, is used as the single-character center feature vector of the overseas character. The single-character center feature vector is stored in a preset feature library as a candidate feature vector.

8. An overseas text recognition device, characterized in that, The device includes: The target character feature sequence determination module is used to extract features from the target overseas text image through the target character recognition network to obtain the target character feature sequence that does not contain character semantic features; The aggregated character feature sequence determination module is used to aggregate each target feature vector in the target character feature sequence based on the similarity between adjacent target feature vectors and the similarity between each target feature vector and the redundant symbol feature vector of a preset redundant symbol, to obtain an aggregated character feature sequence. The matching module is used to match each character feature vector in the aggregated character feature sequence with candidate feature vectors in a preset feature library, and determine the overseas character corresponding to each character feature vector based on the matching result to obtain the target overseas text.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the overseas text recognition method according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the overseas text recognition method according to any one of claims 1-6.

Images