Text recognition method and device, non-transitory computer-readable storage medium and vehicle

Abstract

The application provides a text recognition method and device, a storage medium and a vehicle. The method comprises the following steps: determining a first probability of a character in a to-be-recognized text in a preset annotation type according to a preset NLP model; determining a second probability of the character in the to-be-recognized text in the preset annotation type according to a preset CV network model and the first probability; outputting target annotation information according to the first probability and the second probability; and performing sentence segmentation recognition on the to-be-recognized text according to the target annotation information. The application determines the target annotation information of the to-be-recognized text to perform sentence segmentation on the to-be-recognized text, thereby improving the accuracy of sentence segmentation of the to-be-recognized text, improving the accuracy of text recognition, and facilitating accurate voice control according to the text recognition result.

Description

Technical Field

[0001] This invention relates to the field of intelligent recognition technology, and in particular to a text recognition method and apparatus, a non-transitory computer-readable storage medium, and a vehicle. Background Technology

[0002] With the increasing popularity of artificial intelligence, its applications have permeated all aspects of our lives. NLP (Natural Language Processing) is one of the core areas of artificial intelligence, and it is widely used in fields such as sentiment analysis, question answering systems, automatic summarization, machine translation, speech recognition, chatbots, market forecasting, text classification, and spell checking.

[0003] For text recognition, there are many open-source sentence segmentation models on the market. The recognition method is to divide a long sentence into several short sentences through an algorithm model, and then predict the intent information of each short sentence. However, if the text to be recognized after speech conversion does not have punctuation marks, the text recognition effect is poor when using the above method to segment the sentences, and accurate speech control cannot be performed. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0005] Therefore, one object of the present invention is to provide a text recognition method that improves the accuracy of text recognition and facilitates precise voice control based on the text recognition results.

[0006] Therefore, a second objective of the present invention is to provide a text recognition device.

[0007] Therefore, a third objective of the present invention is to provide a non-transitory computer-readable storage medium.

[0008] Therefore, the fourth object of the present invention is to provide a vehicle.

[0009] To achieve the above objectives, an embodiment of the first aspect of the present invention proposes a text recognition method, the method comprising: determining a first probability of a character in a text to be recognized in a preset annotation type based on a preset NLP model; determining a second probability of a character in the text to be recognized in the preset annotation type based on a preset CV network model and the first probability; outputting target annotation information based on the first probability and the second probability; and performing sentence segmentation recognition on the text to be recognized based on the target annotation information.

[0010] According to the text recognition method of the present invention, by combining a preset NLP model and a preset CV network model, the first probability and the second probability of characters in the text to be recognized in the preset annotation type are determined respectively, and target annotation information is output according to the first probability and the second probability. The text to be recognized is segmented into sentences based on the target annotation information, without the need to segment sentences according to punctuation marks, thereby improving the accuracy of sentence segmentation of the text to be recognized and improving the accuracy of text recognition, which facilitates precise voice control based on the above text recognition results.

[0011] In some embodiments, determining the first probability of a character in a text to be identified in a preset labeling type according to a preset NLP model includes: obtaining a multidimensional feature vector according to a preset BERT model; inputting the multidimensional feature vector into a first classifier to determine a first sub-probability of the character in the text to be identified in the preset labeling type; inputting the multidimensional feature vector into a second classifier to determine a second sub-probability of the character in the text to be identified in the preset labeling type; and determining the first sub-probability and the second sub-probability as the first probability.

[0012] In some embodiments, the first classifier is a CRF classifier, and the second classifier is a softmax classifier.

[0013] In some embodiments, obtaining a multidimensional feature vector based on a preset BERT model includes: determining a numerical sequence of characters in the text to be identified; and inputting the numerical sequence into the preset BERT model to obtain the multidimensional feature vector.

[0014] In some embodiments, determining the first sub-probability and the second sub-probability as the first probability includes: performing probability concatenation on the first sub-probability and the second sub-probability to determine the first probability.

[0015] In some embodiments, determining the second probability of a character in the text to be recognized in the preset annotation type based on a preset CV network model and the first probability includes: determining a concatenated feature vector based on the first probability and a preset function; performing regularization processing on the concatenated feature vector to determine a pixel feature vector within a preset pixel range; and determining the second probability of a character in the text to be recognized in the preset annotation type based on the pixel feature vector and the preset CV network model.

[0016] In some embodiments, the preset CV network model is a preset CNN model, and determining the second probability of a character in the text to be recognized in the preset labeling type based on the pixel feature vector and the preset CV network model includes: inputting the pixel feature vector into the preset CNN model, processing it through a preset convolution kernel to obtain the character feature vector corresponding to the pixel feature vector; and inputting the character feature vector into a third classifier to determine the second probability of a character in the text to be recognized in the preset labeling type.

[0017] In some embodiments, outputting target labeling information based on the first probability and the second probability includes: calculating the variance of the first sub-probability, the second sub-probability, and the second probability; inputting the variance into a fourth classifier to obtain the character type probability of the character in the text to be identified in the preset labeling type, and taking the character type with the highest character type probability as the target labeling information of the character.

[0018] In some embodiments, the text to be identified is segmented and identified based on the target annotation information, including: determining the start annotation character and the end annotation character in the target annotation information; segmenting the text to be identified based on the start annotation character and the end annotation character; and performing segmentation and identification based on the segmented text to be identified.

[0019] To achieve the above objectives, a second aspect of the present invention provides a text recognition device, comprising: a first determining module, configured to determine a first probability of a character in a text to be recognized in a preset labeling type based on a preset NLP model; a second determining module, configured to determine a second probability of a character in the text to be recognized in the preset labeling type based on a preset CV network model and the first probability; an output module, configured to output target labeling information based on the first probability and the second probability; and a recognition module, configured to perform sentence segmentation recognition on the text to be recognized based on the target labeling information.

[0020] According to the text recognition device of the present invention, by combining a preset NLP model and a preset CV network model, the first probability and the second probability of a character in the text to be recognized in a preset annotation type are determined respectively, and target annotation information is output according to the first probability and the second probability. The text to be recognized is segmented into sentences based on the target annotation information, without the need for sentence segmentation based on punctuation marks, thereby improving the accuracy of sentence segmentation of the text to be recognized and improving the accuracy of text recognition, which facilitates precise voice control based on the above text recognition results.

[0021] To achieve the above objectives, a third aspect of the present invention provides a non-transitory computer-readable storage medium storing a text recognition program, which, when executed by a processor, implements the text recognition method as described in the above embodiments.

[0022] To achieve the above objectives, a fourth aspect of the present invention provides a vehicle including the text recognition device described in the above embodiments.

[0023] According to embodiments of the present invention, a vehicle uses a preset NLP model and a preset CV network model to determine the first probability and second probability of characters in the text to be recognized in a preset annotation type, respectively. Target annotation information is output based on the first and second probabilities. The text to be recognized is then segmented into sentences based on the target annotation information, eliminating the need for sentence segmentation based on punctuation marks. This improves the accuracy of sentence segmentation and text recognition, facilitating precise voice control based on the text recognition results. Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0025] Figure 1 This is a flowchart of a text recognition method according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart of a text recognition method according to a specific embodiment of the present invention;

[0027] Figure 3 This is a block diagram of a text recognition device according to an embodiment of the present invention;

[0028] Figure 4 This is a block diagram of a vehicle according to an embodiment of the present invention. Detailed Implementation

[0029] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. The embodiments of the present invention are described in detail below.

[0030] For longer texts to be recognized, NLP (Neural Language Processing) artificial intelligence can accurately analyze and predict the intent contained in the text, which presents a challenge for computers. The ability to accurately infer all the semantic information contained in the text to be detected has profound significance at the NLP application level.

[0031] However, most commonly used sentence segmentation models are based on punctuation marks in the text to be identified. If there are no punctuation marks in the text to be identified, the text cannot be accurately segmented into sentences. If a sentence segmentation model is used for text recognition, there may be cases where it cannot be accurately recognized or the recognition is incomplete.

[0032] Therefore, the text recognition method of this invention combines NLP algorithms and CV (Computer Vision) algorithms to identify the beginning and end of each short sentence in the text to be recognized, thereby segmenting the text and achieving sentence segmentation. This does not rely on punctuation marks in the text to be recognized, thus improving the accuracy of text recognition and facilitating corresponding control based on the recognized text.

[0033] The following is for reference. Figure 1 and Figure 2 The text recognition method of this invention will be illustrated by example, such as... Figure 1 The text recognition method of the present invention shown in this embodiment includes at least steps S1-S4.

[0034] Step S1: Determine the first probability of a character in the text to be recognized in the preset annotation type based on the preset NLP model.

[0035] The preset annotation type is the type of annotation required for each character in the text to be recognized. The preset annotation types include start annotation type, end annotation type and other annotation types. For example, the start annotation type is S, the end annotation type is E, and other annotation types are O.

[0036] In this embodiment, after receiving a voice control command, the voice control command is converted into text to be recognized. After obtaining the text to be recognized, the text to be recognized is input into a preset NLP model. The model is trained to obtain the first probability of each character in the preset labeling type. For example, if the text to be recognized is "open car window", the first probability of the characters "open car window" in the preset labeling type is determined by training the model on the above text. By determining the probability of each character in the text to be recognized in the preset labeling type, it is convenient to optimize the preset NLP model according to the above probability and improve the accuracy of characters in the preset labeling type.

[0037] Step S2: Based on the preset CV network model and the first probability, determine the second probability of the characters in the text to be recognized in the preset annotation type.

[0038] The second probability requires determining the probability of a character in a preset labeling type again based on a preset CV network model after determining the first probability of the character in the preset labeling type, so as to obtain the second probability of the character in the text to be recognized in the preset labeling type.

[0039] In an embodiment, after determining the first probability of each character in the text to be recognized for a preset annotation type, in order to improve the accuracy of character recognition, the above first probability is processed again. For example, the first probability is input into a preset function to obtain a feature vector, and the feature vector is used as an input parameter and input into a preset CV network model to obtain the second probability of the character in the text to be recognized for the preset annotation type.

[0040] It can be understood that when determining the second probability, by combining a preset NLP model and a preset CV network model to determine the probability of a character in a preset annotation type, the maximum probability that each character belongs to a certain annotation type in the preset annotation type can be obtained, thereby improving the accuracy of character category recognition.

[0041] Step S3: Output target annotation information according to the first probability and the second probability.

[0042] Among them, the target annotation information is the character type to which the character with the highest probability of the character type belongs. For example, if the character type with the highest probability of the character "打" in "打开天窗关闭车窗" is S, then S is used as the target annotation information of "打". Similarly, the remaining target annotation information can be obtained. For example, the target annotation information corresponding to "打开天窗关闭车窗" is SOOESOOE.

[0043] In an embodiment, in order to determine the target annotation information of the text to be recognized, it is necessary to calculate the first probability and the second probability of the characters in the text to be recognized for the preset annotation type, and calculate the variance of the first probability and the second probability, so as to obtain the character type to which the character with the highest probability of the character type in the preset annotation type belongs. For example, if it is determined according to the results of the two probabilities that the probability of the character "打" being S is \(0.9\), then the output result of "打" being S is output.

[0044] Step S5: Recognize the text to be recognized according to the target annotation information.

[0045] In an embodiment, the target annotation information is output, that is, the standard information corresponding to each character in the text to be recognized is one of the preset annotation types. For example, if the target annotation information is SOOESOOE, then the text to be recognized is recognized according to the above target annotation information. For example, the text to be recognized "打开天窗关闭车窗" is segmented according to SOOESOOE, and it is determined that the above text to be recognized is two segments. There is no need to use punctuation marks to recognize the above text to be recognized. By recognizing the start position and the end position of the text to be recognized, the accuracy of text recognition can be improved.

[0046] According to the text recognition method of the present invention, by combining a preset NLP model and a preset CV network model, the first probability and the second probability of characters in the text to be recognized in the preset annotation type are determined respectively, and target annotation information is output according to the first probability and the second probability. The text to be recognized is segmented into sentences based on the target annotation information, without the need to segment sentences according to punctuation marks, thereby improving the accuracy of sentence segmentation of the text to be recognized and improving the accuracy of text recognition, which facilitates precise voice control based on the above text recognition results.

[0047] In some embodiments, determining the first probability of a character in the text to be identified in a preset labeling type according to a preset NLP model includes: inputting the character in the text to be identified into a preset BERT model to obtain a multidimensional feature vector; inputting the multidimensional feature vector into a first classifier to determine the first sub-probability of the character in the text to be identified in the preset labeling type; inputting the multidimensional feature vector into a second classifier to determine the second sub-probability of the character in the text to be identified in the preset labeling type; and determining the first sub-probability and the second sub-probability as the first probability.

[0048] In this embodiment, after acquiring the text to be recognized, the text is input into a preset BERT model. After passing through the encoding layer of the preset BERT model, a set of multi-dimensional feature vectors is obtained. These multi-dimensional feature vectors are then input into a first classifier and a second classifier, respectively, to obtain a first sub-probability and a second sub-probability. By simultaneously processing the multi-dimensional feature vectors through the two classifiers for training and calculation, the first sub-probability and the second sub-probability of the characters in the text to be recognized in the preset labeling type can be obtained. The first sub-probability and the second sub-probability are then concatenated to obtain the concatenated first probability. By concatenating the first sub-probability and the second sub-probability, the type probability of the characters in the text to be recognized can be optimized, thereby improving the accuracy of the character type probability.

[0049] For example, after obtaining the text to be recognized, each character in the text is digitized. For example, after obtaining the text to be recognized, "open the car window and set the air conditioner temperature to 25 degrees", each character in the text to be recognized is digitized to obtain an array sequence, with each character corresponding to a number. For example, the number sequence corresponding to the text to be recognized is [167 147 161 100 156 178 176 164 157 154 174 138 185 146].

[0050] After obtaining the above numerical sequence, the numerical sequence is input into a preset BERT model to obtain a set of multi-dimensional feature vectors. After obtaining a set of multi-dimensional feature vectors, the multi-dimensional feature vectors are input into the first classifier, such as input into a CRF classifier (e.g., the CRF layer of NER (Named Entity Recognition)) to determine the first probability of each character in the preset label type, such as determining the first sub-probability of the characters in the above text to be recognized in the preset label type.

[0051] Meanwhile, the multidimensional feature vectors are input into the second classifier, such as the softmax classifier. That is, the multidimensional feature vectors are input into the above classifier, and through training and calculation, the second probability of each character in the preset label type is determined.

[0052] In some embodiments, determining the first sub-probability and the second sub-probability as the first probability includes: concatenating the first sub-probability and the second sub-probability to determine the first probability.

[0053] In this embodiment, when determining the first probability, the first probability and the second probability are concatenated to obtain the first probability, which is used to determine the first probability of the character in the preset labeling type. The above probability is calculated by a preset activation function to obtain the concatenated feature vector. By concatenating the probabilities and calculating the concatenated feature vector corresponding to the first probability, the prediction error of the first sub-probability and the second sub-probability of the character in the preset labeling type can be reduced. By performing function calculation on the concatenated first probability, the difference and damage of the results are reduced, and it participates in the model penalty.

[0054] For example, after the air conditioner temperature is set to 25 degrees and a multi-dimensional feature vector is obtained, the first and second sub-probabilities of each character in the text to be identified are obtained. For example, taking one character as an example, after processing by the preset BERT model and the first classifier, the first sub-probability of belonging to S is determined to be 0.5. At the same time, after processing by the preset BERT model and the second classifier, the probability of belonging to S is determined to be 0.95. The above probabilities are concatenated to determine the first probability of the character in the preset label type.

[0055] Understandably, processing multidimensional feature vectors multiple times can improve the accuracy of determining the preset label type to which a character belongs.

[0056] In some embodiments, determining the second probability of a character in the text to be recognized in a preset annotation type based on a preset CV network model and a first probability includes: determining a concatenated feature vector based on the first probability and a preset function; performing regularization processing on the concatenated feature vector to determine pixel feature vectors within a preset pixel range; and determining the second probability of a character in the text to be recognized in a preset annotation type based on the pixel feature vectors and the preset CV network model.

[0057] In this embodiment, after obtaining the first probability, the first probability is input into a preset function to obtain a concatenated feature vector, and the concatenated feature vector is regularized to a pixel feature vector within a preset pixel range to obtain a pixel feature vector, which is in a normal distribution.

[0058] For example, after obtaining the spliced ​​feature vector, the spliced ​​feature vector is regularized to a preset pixel range, such as (0-255), so that the spliced ​​feature vector is in a normal distribution within the image pixel range, resulting in a regularized pixel feature vector, which makes the spliced ​​feature vector have a regular distribution and is not too scattered.

[0059] After determining the pixel feature vector, the pixel feature vector is input into the preset CV network model to determine the second probability of the character in the text to be recognized in the preset annotation type.

[0060] Specifically, the preset CV network model can be a preset CNN model. Based on the pixel feature vector and the preset CV network model, when determining the second probability of a character in the text to be recognized in the preset labeling type, the pixel feature vector is input into the preset CNN model and processed by the preset convolution kernel to obtain the character feature vector corresponding to the pixel feature vector. The character feature vector is then input into the third classifier to determine the second probability of a character in the text to be recognized in the preset labeling type.

[0061] Understandably, by processing pixel feature vectors with preset convolutional kernels, character feature vectors are obtained corresponding to the pixel feature vectors. The aim is to focus on the start and end positions of the entire regularized pixel feature vector, making the classification result of each character more similar to the index corresponding to the extraction result of the named entity recognition model. After obtaining the character feature vector, the character feature vector is expanded through a fully connected layer in deep learning and input into a third classifier, such as a softmax classifier, to obtain the second probability of each character in a preset label type, for example, the maximum probability that each character belongs to a SEO intent. By determining the start and end positions of the entire sequence in the text to be recognized, the classification result of each character is made closer to the index corresponding to the extraction result of NER (Named Entity Recognition).

[0062] In this method, by using the naming recognition model prediction and the neural network loss of the third classifier, errors in the extraction process and errors between the classifier and the preset neural network can be corrected and penalized. Each loss is configured with weight parameters, and the weight parameters correspond to the training of the preset neural network, which can improve the accuracy of determining the character type.

[0063] In some embodiments, outputting target labeling information based on a first probability and a second probability includes: calculating the variance of the first sub-probability, the second sub-probability, and the second probability; inputting the variance into a fourth classifier to obtain the character type probability of the character in the text to be identified in a preset labeling type, and taking the character type with the highest character type probability as the target labeling information of the character.

[0064] In this embodiment, after determining the first sub-probability, the second sub-probability, and the second probability, the variance of the three is calculated and input into the fourth classifier to obtain the output value. The output value is the character type with the highest probability in the text to be identified, and the character type with the highest probability is used as the target labeling information of the character, such as the target labeling information of the SOE corresponding to the text to be identified.

[0065] Calculating the variance of the three factors allows the error to fluctuate within the normal distribution range when the probability deviates, preventing excessive error from causing inaccurate output values ​​for labeled characters.

[0066] By calculating the deviation between the output value and the preset annotation type, and using this deviation as a loss in a preset neural network model for correction, the error is reduced until the annotated character output value is within a preset variance range. By calculating the error and allowing it to vary within a normal distribution range, the model becomes more persuasive and robust, thus improving the accuracy of the annotated character output value.

[0067] In some embodiments, the process of segmenting and identifying the text to be identified based on the target annotation information includes: determining the start annotation character and the end annotation character in the target annotation information; segmenting the text to be identified based on the start annotation character and the end annotation character; and performing segmentation and identification based on the segmented text to be identified.

[0068] In this embodiment, after determining the target annotation information, such as the annotation information of SOOOE, the start annotation character and the end annotation character in the target annotation information are determined. The initial position and the end position in the text to be recognized are determined according to the annotation characters. The text to be recognized is segmented according to the initial position and the end position to achieve the segmentation recognition of the text to be recognized.

[0069] The following is for reference. Figure 2 The text recognition method described in the embodiments of the present invention, such as Figure 2 The diagram shown is a flowchart of a text recognition method according to an embodiment of the present invention.

[0070] Step S11: Obtain the text to be recognized corresponding to the voice command.

[0071] Step S12: Determine the numerical sequence of characters in the text to be recognized, input the numerical sequence into the preset BERT model, and obtain a multidimensional feature vector.

[0072] Step S13: Input the multidimensional feature vector into the first classifier to determine the first sub-probability of the character in the text to be identified in the preset label type.

[0073] Step S14: Input the multidimensional feature vector into the second classifier to determine the second sub-probability of the characters in the text to be identified in the preset label type.

[0074] Step S15: Determine the first sub-probability and the second sub-probability as the first probability.

[0075] Step S16: Determine the concatenated feature vector based on the first probability and the preset function.

[0076] Step S17: Perform regularization on the spliced ​​feature vector to determine the pixel feature vector within the preset pixel range.

[0077] Step S18: Determine the second probability of the characters in the text to be recognized in the preset annotation type based on the pixel feature vector and the preset CV network model.

[0078] Step S19: Calculate the variance of the first sub-probability, the second sub-probability, and the second probability.

[0079] Step S20: Input the variance into the fourth classifier to obtain the character type probability of the character in the text to be identified in the preset label type, and take the character type with the highest character type probability as the target label information of the character.

[0080] Step S21: Recognize the text to be recognized based on the target annotation information.

[0081] In summary, by combining a pre-set NLP model and a pre-set CV network model, the first and second probabilities of characters in the text to be recognized within a pre-set annotation type are determined respectively. Target annotation information is then output based on these probabilities. Sentence segmentation of the text to be recognized is performed based on this target annotation information, eliminating the need for punctuation-based segmentation. This improves the accuracy of sentence segmentation and text recognition, facilitating precise voice control based on the text recognition results. Furthermore, by treating each character in the text to be recognized as a pixel in an image, a convolutional neural network focuses on the beginning and end of each short phrase in the text to achieve text recognition. Simultaneously, NLP classification and named entity recognition are used to obtain the beginning and end classification and slot information. The errors in the inference results of these three methods are penalized, ensuring that the error between the final inference result and the true result falls within the range of a minimum normal distribution. Compared to simple classification, this method exhibits smaller bias and higher accuracy.

[0082] The text recognition device according to an embodiment of the present invention is described below.

[0083] like Figure 3 As shown, the text recognition device 2 of this embodiment includes a first determining module 21, a second determining module 22, an output module 23, and a recognition module 24, wherein...

[0084] The first determining module 21 is used to determine the first probability of a character in the text to be identified in a preset labeling type according to a preset NLP model; the second determining module 22 is used to determine the second probability of a character in the text to be identified in the preset labeling type according to a preset CV network model and the first probability; the output module 23 is used to output target labeling information according to the first probability and the second probability; the recognition module 24 is used to perform sentence segmentation recognition on the text to be identified according to the target labeling information.

[0085] The text recognition device 2 according to an embodiment of the present invention, by combining a preset NLP model and a preset CV network model, determines the first probability and second probability of characters in the text to be recognized in a preset annotation type, respectively, and outputs target annotation information based on the first and second probabilities. The text to be recognized is then segmented into sentences based on the target annotation information, eliminating the need for sentence segmentation based on punctuation marks. This improves the accuracy of sentence segmentation in the text to be recognized, thereby enhancing the accuracy of text recognition and facilitating precise voice control based on the text recognition results.

[0086] The vehicle according to an embodiment of the present invention is described below.

[0087] like Figure 4 As shown, the vehicle 3 in this embodiment of the invention includes the text recognition device 2 described in the above embodiment.

[0088] According to the embodiment of the present invention, the vehicle 3 uses a preset NLP model and a preset CV network model to determine the first probability and the second probability of the characters in the text to be recognized in the preset annotation type, respectively, and outputs target annotation information based on the first probability and the second probability. The text to be recognized is segmented into sentences based on the target annotation information, without the need to segment sentences based on punctuation marks, thereby improving the accuracy of sentence segmentation of the text to be recognized and improving the accuracy of text recognition, which facilitates precise voice control based on the above text recognition results.

[0089] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0090] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A text recognition method, characterized in that, include: Based on the preset NLP model, determine the first probability of the characters in the text to be recognized in the preset annotation type; Based on the preset computer vision network model and the first probability, determine the second probability of the character in the text to be recognized in the preset annotation type; Output target labeling information based on the first probability and the second probability; Sentence segmentation and recognition are performed on the text to be identified based on the target annotation information; The step of determining the second probability of a character in the text to be recognized in the preset annotation type based on a preset computer vision network model and the first probability includes: The concatenated feature vector is determined based on the first probability and the preset function; The spliced ​​feature vector is regularized to determine the pixel feature vector within a preset pixel range; Based on the pixel feature vector and the preset computer vision network model, the second probability of the character in the text to be recognized in the preset annotation type is determined.

2. The text recognition method according to claim 1, characterized in that, Based on the preset NLP model, determine the first probability of a character in the text to be recognized belonging to a preset annotation type, including: A multidimensional feature vector is obtained based on the preset Bert model; The multidimensional feature vector is input into the first classifier to determine the first sub-probability of the characters in the text to be identified in the preset labeling type; The multidimensional feature vector is input into a second classifier to determine the second sub-probability of the characters in the text to be identified in a preset labeling type; The first sub-probability and the second sub-probability are determined as the first probability.

3. The text recognition method according to claim 2, characterized in that, The first classifier is a CRF classifier, and the second classifier is a softmax classifier.

4. The text recognition method according to claim 2, characterized in that, The multidimensional feature vectors obtained based on the pre-defined BERT model include: Determine the numerical sequence of characters in the text to be identified; The numerical sequence is input into the preset BERT model to obtain the multidimensional feature vector.

5. The text recognition method according to claim 2, characterized in that, Determining the first sub-probability and the second sub-probability as the first probability includes: The first probability is determined by concatenating the first sub-probability and the second sub-probability.

6. The text recognition method according to claim 1, characterized in that, The preset computer vision network model is a preset CNN model. The step of determining the second probability of a character in the text to be recognized in the preset annotation type based on the pixel feature vector and the preset computer vision network model includes: The pixel feature vector is input into the preset CNN model and processed by the preset convolution kernel to obtain the character feature vector corresponding to the pixel feature vector; The character feature vector is input into a third classifier to determine the second probability of the character in the text to be identified in the preset label type.

7. The text recognition method according to claim 2, characterized in that, Target labeling information is output based on the first probability and the second probability, including: Calculate the first sub-probability, the second sub-probability, and the variance of the second probability; The variance is input into the fourth classifier to obtain the character type probability of the character in the text to be identified in the preset label type, and the character type with the highest character type probability is used as the target label information of the character.

8. The text recognition method according to claim 1, characterized in that, Sentence segmentation and recognition of the text to be identified based on the target annotation information includes: Determine the start and end annotation characters in the target annotation information; The text to be recognized is segmented according to the start label character and the end label character; Sentence segmentation is performed on the segmented text to be identified.

9. A text recognition device, characterized in that, include: The first determination module is used to determine the first probability of a character in the text to be recognized in a preset annotation type based on a preset NLP model. The second determining module is used to determine the second probability of a character in the text to be identified in the preset annotation type based on a preset computer vision network model and the first probability; The output module is used to output target annotation information based on the first probability and the second probability; The recognition module is used to perform sentence segmentation recognition on the text to be recognized based on the target annotation information; When determining the second probability of a character in the text to be identified in the preset annotation type based on a preset computer vision network model and the first probability, the second determining module is specifically used to: determine the concatenated feature vector based on the first probability and a preset function; The spliced ​​feature vector is regularized to determine the pixel feature vector within a preset pixel range; Based on the pixel feature vector and the preset computer vision network model, the second probability of the character in the text to be recognized in the preset annotation type is determined.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores a text recognition program, which, when executed by a processor, implements the text recognition method as described in any one of claims 1-8.

11. A vehicle, characterized in that, include: The text recognition device as described in claim 9.

Images