Text recognition model determination method, text recognition method and device and medium

Abstract

The invention relates to a text recognition model determination method, a text recognition method, a text recognition device and a medium, and the text recognition model determination method comprises the steps of obtaining a pre-trained first text recognition model which is used for executing a first text recognition task, the first text recognition task at least comprises text recognition on image data; determining a first target instruction according to a task demand of a user for a second text recognition task, wherein the first target instruction is used for reflecting an intention of the second text recognition task; constructing first training data according to the first target instruction and a first sample image, and adjusting model parameters of an initial second text recognition model according to the first training data until the model parameters of the second text recognition model meet preset conditions to obtain a trained second text recognition model, the model parameters of the initial second text recognition model are determined according to the model parameters of the first text recognition model.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent education technology, and in particular to a method for determining a text recognition model, a text recognition method, a device, and a medium. Background Art

[0002] In the education sector, AI (Artificial Intelligence) technology solutions have begun to be used to specifically address some needs. Related technologies include text recognition based on OCR (Optical Character Recognition) and rule-based post-processing. However, due to the diverse data types in education, this solution lacks generalizability and versatility. Forcibly covering all data types would incur high costs. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a method for determining a text recognition model, a text recognition method, an apparatus, and a medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for determining a text recognition model is provided, comprising: Obtaining a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; determining a first target instruction according to a task requirement of the user for the second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; First training data is constructed according to the first target instruction and the first sample image, and model parameters of an initial second text recognition model are adjusted according to the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

[0005] Optionally, determining the first target instruction according to the user's task requirements for the second text recognition task includes: determining initial instructions based on the mission requirements; Perform at least one of rewriting, screening, error correction, and classification on the initial instructions to determine a first target instruction, wherein the rewriting includes synonym replacement and / or grammatical transformation, the screening includes removing at least one of repeated instructions, removing instructions with unclear meanings, and removing instructions with an implementation degree less than a threshold, wherein the implementation degree is used to reflect the degree to which the instructions can be implemented, and the error correction includes correcting spelling errors and / or grammatical errors.

[0006] Optionally, the method further includes: The task requirement is determined according to the user's requirement for the second text recognition task when using the text recognition system, the user's requirement for the second text recognition task in different scenarios, and the user's multiple requirements for multiple second text recognition tasks.

[0007] Optionally, the second text recognition model includes a multi-head attention layer, a first additive layer normalization layer, a feedforward network layer, and a second additive layer normalization layer connected in sequence, and adjusting the model parameters of the initial second text recognition model according to the first training data until the model parameters of the second text recognition model meet preset conditions includes: The first training data is input into the initial second text recognition model, and the first parameter of the multi-head attention layer, the second parameter of the first addition and layer normalization layer, the third parameter of the feedforward network layer, and the fourth parameter of the second addition and layer normalization layer are adjusted according to the low-rank decomposition matrix to adjust the model parameters of the initial second text recognition model until the model parameters of the second text recognition model meet preset conditions, thereby obtaining the trained second text recognition model.

[0008] Optionally, the first text recognition model includes a convolutional neural network, an encoding layer, and a decoding layer.

[0009] Optionally, the method further includes: Obtaining a second target instruction, where the second target instruction is used to reflect the intention of the third text recognition task; Second training data is constructed according to a second target instruction and a second sample image, and model parameters of the pre-trained first text recognition model are updated according to the second training data to obtain an updated first text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the updated first text recognition model.

[0010] According to a second aspect of an embodiment of the present disclosure, a text recognition method is provided, including: Obtain the image to be recognized; The image to be recognized is processed according to the trained second text recognition model to obtain a text recognition result.

[0011] According to a third aspect of an embodiment of the present disclosure, a device for determining a text recognition model is provided, comprising: A first acquisition module is configured to acquire a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; A first determining module is configured to determine a first target instruction according to a task requirement of a user for a second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; The first training module is configured to construct first training data based on the first target instruction and the first sample image, and adjust the model parameters of the initial second text recognition model based on the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

[0012] According to a fourth aspect of an embodiment of the present disclosure, a device for determining a text recognition model is provided, comprising: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Obtaining a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; determining a first target instruction according to a task requirement of the user for the second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; First training data is constructed according to the first target instruction and the first sample image, and model parameters of an initial second text recognition model are adjusted according to the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

[0013] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for determining the text recognition model provided in the first aspect of the present disclosure are implemented.

[0014] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by constructing an initial second text recognition model using the model parameters of a pre-trained first text recognition model, and constructing first training data using a first sample image and a first target instruction determined according to the user's task requirements for the second text recognition task, the model parameters of the initial second text recognition model are adjusted to obtain a trained second text recognition model, so that the final trained second text recognition model can have a strong instruction-following ability, and can be trained to complete the second text recognition task through a few-sample or zero-sample learning method, thereby greatly improving the training efficiency of the second text recognition model, saving computing resources, and improving OCR recognition accuracy. In addition, by expanding the first target instruction, the second text recognition task that the second text recognition model can perform can be expanded, so that the second text recognition model can adapt to different educational scenarios and needs, and improve the scalability and flexibility of the second text recognition model.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0017] Figure 1 The figure is a flowchart of a method for determining a text recognition model according to an exemplary embodiment.

[0018] Figure 2 FIG. 4 is a schematic diagram of a model structure of a second text recognition model according to an exemplary embodiment.

[0019] Figure 3 The flowchart of updating the model parameters of the first text recognition model is shown according to an exemplary embodiment.

[0020] Figure 4 FIG. 4 is a flowchart showing a method for adjusting parameters of a UPOCR model according to an exemplary embodiment.

[0021] Figure 5 The figure is a flowchart of a text recognition method according to an exemplary embodiment.

[0022] Figure 6 The figure is a block diagram of a device for determining a text recognition model according to an exemplary embodiment.

[0023] Figure 7 The figure is a block diagram of a text recognition device according to an exemplary embodiment.

[0024] Figure 8 The present invention is a block diagram showing a device for determining a text recognition model or for text recognition according to an exemplary embodiment.

[0025] Figure 9 The present invention is a block diagram showing a device for determining a text recognition model or for text recognition according to an exemplary embodiment. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0027] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the corresponding data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0028] As mentioned in the background technology, in the field of education, AI technology solutions have begun to be used to specifically address some needs. In related technologies, text recognition can be performed based on OCR and rule-based post-processing solutions. In addition, related technologies can also perform text recognition through the following solutions: 1) Use vertical field-specific models for text recognition. However, this solution lacks an accurate unified model for multiple scenarios, such as mixed scenarios of word dictation and composition, and comprehensive scenarios of science and liberal arts.

[0029] 2) Using rule engines for text recognition. However, this approach relies on manually written rules to handle various scenarios. While this approach may work well in some specific scenarios, it requires a lot of manpower to write and maintain the rules, and is difficult to adapt to new or unknown scenarios.

[0030] 3) Machine learning: By studying large amounts of labeled data, machine learning models can automatically learn effective features and rules from the data. However, this approach requires a large amount of labeled data and, for complex scenarios, may require the use of complex models and extensive computing resources.

[0031] 4) Expert systems: This approach relies on the knowledge and experience of experts to handle various scenarios. Expert systems can handle some complex problems, but building and maintaining them requires significant manpower and time, and they struggle to adapt to new or unknown scenarios.

[0032] As can be seen, related technologies have the following drawbacks: they consume a large amount of annotation resources, lack good applicability to common scenarios, and have limited model generalization capabilities. They rely heavily on expert prior knowledge, cannot effectively extract data features, cannot couple data characteristics, and cannot effectively build universal models, relying instead on the serial and parallel connection of multiple models.

[0033] To this end, the present disclosure proposes a method for determining a text recognition model, a text recognition method, an apparatus, and a medium, which can improve the generalization ability of the model and quickly train a model that adapts to a variety of scenarios, saving computing resources.

[0034] Figure 1 is a flowchart of a method for determining a text recognition model according to an exemplary embodiment. Figure 1 As shown, the method for determining a text recognition model may include the following steps 110-130.

[0035] Step 110: Obtain a pre-trained first text recognition model. The first text recognition model is used to perform a first text recognition task. The first text recognition task at least includes performing text recognition on image data.

[0036] In some embodiments, the pre-trained first text recognition model may be a pre-trained universal OCR model. For example, the first text recognition model may be a UPOCR (Unit Pixel OCR) model.

[0037] The first text recognition model is used to perform text recognition on image data in the field of education and output text information. For example, the image data may include scanned documents, digital photos, screenshots, etc., and the text information may be text included in the image data.

[0038] In some embodiments, the first text recognition model may include a convolutional neural network, an encoding layer, and a decoding layer. The first text recognition model can convert various first text recognition tasks into sequence prediction. For example, the various first text recognition tasks may include sequential recognition and analysis of different elements such as text, paragraphs, layouts, tables, and formulas.

[0039] Convolutional neural networks are used for image feature extraction, while encoding and decoding layers are used to convert images into sequences. The model structure of the first text recognition model fully captures the semantic information in images and has strong sequence generation capabilities. Furthermore, the first text recognition model of the present disclosure utilizes sequence prediction, enabling the model to achieve excellent performance in text recognition across various scenarios and tasks. It also demonstrates excellent task versatility, enabling the model to handle a variety of text recognition tasks in a unified manner.

[0040] In some embodiments, different prompts can be used to guide the model to complete the training of the first text recognition model. The trained first text recognition model can support paragraph-level document image recognition analysis and output standard formats such as Markdown / HTML / Text. For example, multiple OCR tasks can be defined in the form of sequence prediction, and different prompts can be used to guide the model to complete different OCR tasks. Each task corresponds to a unique prompt. This guidance mechanism allows the model to focus on different goals during the learning process, improving the flexibility and adaptability of text recognition.

[0041] Step 120 : determining a first target instruction according to the user's task requirements for the second text recognition task, where the first target instruction is used to reflect the intention of the second text recognition task.

[0042] The second text recognition task can be different from the first text recognition task. In some embodiments, determining the first target instruction based on the user's task requirements for the second text recognition task includes: determining an initial instruction based on the task requirements; performing at least one of rewriting, screening, error correction, and classification on the initial instruction to determine the first target instruction, wherein the rewriting includes synonym replacement and / or grammatical transformation, the screening includes at least one of removing duplicate instructions, removing instructions with unclear meanings, and removing instructions with a degree of implementation less than a threshold, the degree of implementation being used to reflect the degree to which the instruction can be implemented, and the error correction includes correcting spelling errors and / or grammatical errors.

[0043] In a possible implementation, the initial instruction can be directly determined based on the task requirements, or the initial instruction can be extracted from the task requirements. For example, taking the task requirement of "Please scan this test paper" as an example, "Please scan this test paper" can be directly determined as the initial instruction, or the initial instruction "Scan the test paper" can be extracted from the task requirements.

[0044] In some embodiments, before performing at least one of rewriting, screening, error correction, and classification on the initial instruction to determine the first target instruction, the method further includes: processing the initial instruction using natural language processing technology. The natural language processing technology is used to convert the initial instruction into a form that can be understood by the system.

[0045] In some embodiments, synonym replacement can be performed on the initial instruction using a synonym library or word vector model. Synonym replacement can replace words in the initial instruction with words with similar meanings but different expressions. For example, "scan" included in the initial instruction can be replaced with "take a photo."

[0046] In a possible implementation, grammatical transformation involves changing the sentence structure of the initial instruction. For example, taking the initial instructions "Please scan this test paper" and "Identify the answers," after grammatical transformation, they can be converted to "Scan this test paper" and "Read the answers," respectively. The disclosed embodiments rewrite the initial instructions using synonym replacement and grammatical transformation techniques to generate more instruction variants.

[0047] In a possible implementation, error correction can be performed using rules or machine learning models. By using rules or machine learning models for error correction, it is possible to automatically detect and correct problems such as spelling errors and grammatical errors.

[0048] In some embodiments, an instruction with an implementation degree less than a threshold value can indicate that the instruction is difficult to implement. In a possible implementation, the first target instruction can also be screened through manual review and testing to ensure that the first target instruction can be executed accurately and the output result meets expectations.

[0049] The embodiment of the present disclosure screens the initial instructions by removing at least one of duplicate instructions, instructions with unclear meanings, and instructions with implementation levels less than a threshold, thereby obtaining high-quality first target instructions.

[0050] In some embodiments, the classification may be based on the category of the instruction. For example, the instruction may be classified into a scanning category and a recognition category, etc. In a possible implementation, the instruction may be labeled based on the classification result of the instruction.

[0051] In some embodiments, the first target instruction may be an instruction in a preset instruction format. For example, the preset instruction format may be designed based on multiple aspects such as consistency, simplicity, semantic clarity, and user-friendliness. Consistency is used to ensure that all OCR-related command formats are consistent, for example, instructions such as "scan test paper" and "recognize answers" use a unified format. Simplicity is used to make instructions as short as possible, for example, "start scanning" and "export results." Semantic clarity is used to clarify the function of the instruction to avoid confusion, such as "recognize table area" and "mark handwritten part." User-friendliness takes into account that users may have different technical backgrounds, and tries to use simple and clear instructions, for example, "upload picture" and "download PDF".

[0052] In some embodiments, the method further includes determining task requirements based on a user's requirements for a second text recognition task when using the text recognition system, the user's requirements for the second text recognition task in different scenarios, and the user's multiple requirements for multiple second text recognition tasks. The second text recognition task can be related to different test paper formats and different subject content. The different scenarios can include: automatically grading test papers, recognizing and evaluating students' handwritten assignments, and transcribing classroom handouts.

[0053] For example, when using a text recognition system, user requirements for a second text recognition task may include: "automatically grading multiple-choice questions" and "recognizing formulas." User requirements for a second text recognition task in different scenarios may include: "quickly calculating scores after an exam" and "recognizing complex charts." User requirements for various second text recognition tasks may include: "recognizing and grading an entire exam paper."

[0054] The disclosed embodiment determines task requirements based on the user's needs for the second text recognition task when using the text recognition system, the user's needs for the second text recognition task in different scenarios, and the user's multiple needs for multiple second text recognition tasks, and can obtain a large number of task requirements, thereby expanding the number of first target instructions.

[0055] Furthermore, by setting up secondary text recognition tasks related to different test paper styles and subject content, the disclosed embodiments enable the ultimately trained secondary text recognition model to achieve more accurate educational data analysis, thereby helping educational institutions or teachers better understand students' learning status, thereby improving teaching effectiveness and learning efficiency. Furthermore, the secondary text recognition model can automatically adapt to different test paper styles and subject content, while maintaining high efficiency and accuracy while reducing processing costs.

[0056] In a possible implementation, questionnaires and user interviews may be used to obtain at least one of the user's needs for the second text recognition task when using the text recognition system, the user's needs for the second text recognition task in different scenarios, and the user's multiple needs for multiple second text recognition tasks.

[0057] In some embodiments, the method further comprises: adjusting the first target instruction according to user feedback. The disclosed embodiment adjusts the first target instruction according to user feedback, which can continuously optimize the instruction library, add new functions or improve the description of existing functions.

[0058] Step 130: construct first training data based on the first target instruction and the first sample image, and adjust model parameters of the initial second text recognition model based on the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model. The model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

[0059] The disclosed embodiment constructs an initial second text recognition model using the model parameters of a pre-trained first text recognition model, and constructs first training data using a first sample image and a first target instruction determined by the user according to the task requirements of the second text recognition task, thereby adjusting the model parameters of the initial second text recognition model to obtain a trained second text recognition model, so that the final trained second text recognition model has a strong ability to follow instructions, and can be trained to complete the second text recognition task through a few-sample or zero-sample learning method, thereby greatly improving the training efficiency of the second text recognition model, saving computing resources, and improving OCR recognition accuracy. In addition, by expanding the first target instruction, the second text recognition task that the second text recognition model can perform can be expanded, so that the second text recognition model can adapt to different educational scenarios and needs, and improve the scalability and flexibility of the second text recognition model.

[0060] In some embodiments, the first training data may also carry labels, which may be annotated text recognition results. Based on the labeled first training data, supervised training is used to adjust the model parameters of the initial second text recognition model until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model. For details about supervised training, please refer to the relevant art and will not be further described here.

[0061] In a possible implementation, the second text recognition model includes a multi-head attention layer, a first additive and layer normalization layer, a feedforward network layer, and a second additive and layer normalization layer connected in sequence, and the model parameters of the initial second text recognition model are adjusted according to the first training data until the model parameters of the second text recognition model meet preset conditions, including: inputting the first training data into the initial second text recognition model, and adjusting the first parameter of the multi-head attention layer, the second parameter of the first additive and layer normalization layer, the third parameter of the feedforward network layer, and the fourth parameter of the second additive and layer normalization layer according to the low-rank decomposition matrix to adjust the model parameters of the initial second text recognition model until the model parameters of the second text recognition model meet the preset conditions, thereby obtaining a trained second text recognition model.

[0062] For example, Figure 2As shown, the second text recognition model 200 includes a multi-head attention layer 210, a first addition layer normalization layer 220, a feedforward network layer 230, and a second addition layer normalization layer 240 connected in sequence. In a possible embodiment, the second text recognition model 200 is further connected to an embedding layer 250, which is used to encode the input first training data to obtain an encoding vector. Figure 2 As shown, by adjusting the parameters of each layer, the model parameters of the initial second text recognition model can be adjusted.

[0063] In a possible implementation, the parameters of each layer can be adjusted by the following formula (1): (1) in, represents the adjusted parameters, Indicates the parameter to be adjusted, represents the low-rank decomposition matrix.

[0064] The disclosed embodiment approximates the parameter update of each layer by adding a low-rank decomposition matrix to the parameter matrix, thereby reducing the parameters that need to be trained to adapt to downstream tasks.

[0065] Figure 3 FIG. 1 is a flow chart showing a method for updating model parameters of a first text recognition model according to an exemplary embodiment. Figure 3 As shown, the process may include the following steps 310 and 320.

[0066] Step 310: Acquire a second target instruction, where the second target instruction is used to reflect the intention of the third text recognition task.

[0067] The second target instruction can be determined based on a preset task, which can be determined based on the user's specific needs. For example, if a test paper contains both handwritten and printed text and only print text is required to be recognized, the second target instruction "print text recognition" can be set for this task.

[0068] In some embodiments, the third text recognition task includes multiple tasks, and the second target instruction can also be used to reflect the priority recognition of the third text recognition task among the multiple third text recognition tasks.

[0069] Step 320: construct second training data based on the second target instruction and the second sample image, and update the model parameters of the pre-trained first text recognition model based on the second training data to obtain an updated first text recognition model. The model parameters of the initial second text recognition model are determined based on the model parameters of the updated first text recognition model.

[0070] After pre-training the first text recognition model, the embodiment of the present disclosure constructs second training data based on the second target instruction and the second sample image, and updates the model parameters of the pre-trained first text recognition model based on the second training data. By adjusting the model parameters of the first text recognition model, the first text recognition model can be adapted to a specific third text recognition task.

[0071] In a possible implementation, the model parameters of the pre-trained first text recognition model can be updated by the instruction controller. For example, taking the first text recognition model as a UPOCR model, the following can be done: Figure 4 The parameters of the UPOCR model are adjusted in the manner shown.

[0072] Figure 4 The illustrated process includes steps 410-450. Step 410 is data preparation and preprocessing. Data preparation may include preparing an image dataset in the field of education, which is used to train and test UPOCR. Step 420 is UPOCR model training. Step 420 is used to obtain a pre-trained UPOCR model. Step 430 is UPOCR model parameter determination. Step 430 is used to determine the model parameters of the pre-trained UPOCR model. Step 440 is used to control the instruction controller, which is used to adjust the model parameters of the pre-trained UPOCR model based on the second training data. Step 450 is used to evaluate the effectiveness of the model. For example, the effectiveness evaluation can be completed by running the model on a test set and comparing its output with the actual results.

[0073] In some embodiments, Figure 4 The process shown also includes step 460, which is to adjust the parameters of the UPOCR model. For details on the adjustment, please refer to the above Figure 1 The related descriptions are not repeated here, and the second text recognition model can be obtained through adjustment.

[0074] Figure 5 FIG. 1 is a flowchart of a text recognition method according to an exemplary embodiment. Figure 5 As shown, the process includes the following steps 510 and 520.

[0075] Step 510: Obtain an image to be recognized.

[0076] Step 520: Process the image to be recognized according to the trained second text recognition model to obtain a text recognition result.

[0077] In some embodiments, the trained second text recognition model is obtained according to the aforementioned method for determining the text recognition model. Figure 1-Figure 3 The related descriptions will not be repeated here.

[0078] The embodiment of the present disclosure constructs an initial second text recognition model by using the model parameters of the pre-trained first text recognition model, and constructs the first training data using the first sample image and the first target instruction determined according to the task requirements of the user for the second text recognition task, to adjust the model parameters of the initial second text recognition model to obtain a trained second text recognition model, so that the second text recognition model finally trained can have a strong instruction-following ability, and can be trained to complete the second text recognition task by means of few-sample or zero-sample learning, thereby greatly improving the training efficiency of the second text recognition model, saving computing resources, and improving the OCR recognition accuracy. In addition, by expanding the first target instruction, the second text recognition task that the second text recognition model can perform can be expanded, so that the second text recognition model can adapt to different educational scenarios and needs, and improve the scalability and flexibility of the second text recognition model. This can greatly improve the accuracy of the text recognition results of the present disclosure.

[0079] Figure 6 FIG. 1 is a block diagram of a device for determining a text recognition model according to an exemplary embodiment. Figure 6 , the text recognition model determination device 600 may include: A first acquisition module 610 is configured to acquire a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; A first determining module 620 is configured to determine a first target instruction according to a task requirement of the user for the second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; The first training module 630 is configured to construct first training data based on the first target instruction and the first sample image, and adjust the model parameters of the initial second text recognition model based on the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, where the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

[0080] Optionally, the first determining module 620 is further configured to: determining initial instructions based on the mission requirements; Perform at least one of rewriting, screening, error correction, and classification on the initial instructions to determine a first target instruction, wherein the rewriting includes synonym replacement and / or grammatical transformation, the screening includes removing at least one of repeated instructions, removing instructions with unclear meanings, and removing instructions with an implementation degree less than a threshold, wherein the implementation degree is used to reflect the degree to which the instructions can be implemented, and the error correction includes correcting spelling errors and / or grammatical errors.

[0081] Optionally, the apparatus 600 further includes: The second determination module is configured to determine the task requirement based on the user's requirements for the second text recognition task when using the text recognition system, the user's requirements for the second text recognition task in different scenarios, and the user's multiple requirements for multiple second text recognition tasks.

[0082] Optionally, the second text recognition model includes a multi-head attention layer, a first addition and layer normalization layer, a feedforward network layer, and a second addition and layer normalization layer connected in sequence, and the first training module 630 is further configured to: The first training data is input into the initial second text recognition model, and the first parameter of the multi-head attention layer, the second parameter of the first addition and layer normalization layer, the third parameter of the feedforward network layer, and the fourth parameter of the second addition and layer normalization layer are adjusted according to the low-rank decomposition matrix to adjust the model parameters of the initial second text recognition model until the model parameters of the second text recognition model meet preset conditions, thereby obtaining the trained second text recognition model.

[0083] Optionally, the first text recognition model includes a convolutional neural network, an encoding layer, and a decoding layer.

[0084] Optionally, the apparatus 600 further includes: a second acquisition module configured to acquire a second target instruction, where the second target instruction is used to reflect the intention of the third text recognition task; The second training module is configured to construct second training data based on the second target instruction and the second sample image, and update the model parameters of the pre-trained first text recognition model based on the second training data to obtain an updated first text recognition model, and the model parameters of the initial second text recognition model are determined based on the model parameters of the updated first text recognition model.

[0085] Regarding the device for determining the text recognition model in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method and will not be elaborated on here.

[0086] Figure 7 FIG. 1 is a block diagram of a text recognition device according to an exemplary embodiment. Figure 7 As shown, the text recognition device 700 includes: The third acquisition module 710 is configured to acquire an image to be recognized; The processing module 720 is configured to process the image to be recognized according to the trained second text recognition model to obtain a text recognition result.

[0087] Regarding the text recognition device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0088] The present disclosure also provides a device for determining a text recognition model, comprising: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Obtaining a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; determining a first target instruction according to a task requirement of the user for the second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; First training data is constructed according to the first target instruction and the first sample image, and model parameters of an initial second text recognition model are adjusted according to the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

[0089] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the method for determining a text recognition model and the steps of the text recognition method provided by the present disclosure are implemented.

[0090] Figure 8 8 is a block diagram illustrating an apparatus 800 for determining a text recognition model or performing text recognition, according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0091] Reference Figure 8, the apparatus 800 may include one or more of the following components: a first processing component 802 , a first memory 804 , a first power supply component 806 , a multimedia component 808 , an audio component 810 , a first input / output interface 812 , a sensor component 814 , and a communication component 816 .

[0092] The first processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The first processing component 802 may include one or more first processors 820 to execute instructions to complete all or part of the steps of the above-mentioned text recognition model determination method or text recognition method. In addition, the first processing component 802 may include one or more modules to facilitate interaction between the first processing component 802 and other components. For example, the first processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the first processing component 802.

[0093] The first memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The first memory 804 can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0094] The first power supply component 806 provides power to various components of the device 800. The first power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0095] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, the multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a capture mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and the rear-facing camera can have a fixed optical lens system or have focal length and optical zoom capabilities.

[0096] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, or a voice recognition mode. The received audio signals may be further stored in the first memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0097] The first input / output interface 812 provides an interface between the first processing component 802 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0098] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor assembly 814 can also detect changes in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0099] The communication component 816 is configured to facilitate wired or wireless communication between the apparatus 800 and other devices. The apparatus 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0100] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-mentioned text recognition model determination method or text recognition method.

[0101] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a first memory 804 including instructions. The instructions may be executed by the first processor 820 of the apparatus 800 to perform the above-described method for determining a text recognition model or text recognition method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0102] Figure 9 1 is a block diagram of an apparatus 900 for determining a text recognition model or for text recognition according to an exemplary embodiment. For example, the apparatus 900 may be provided as a server. Figure 9 The apparatus 900 includes a second processing component 922, which further includes one or more processors, and a memory resource represented by a second memory 932 for storing instructions executable by the second processing component 922, such as an application. The application stored in the second memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the second processing component 922 is configured to execute the instructions to perform the above-mentioned method for determining a text recognition model or text recognition.

[0103] The device 900 may further include a second power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and a second input / output interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server 2003. TM , Mac OSX TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0104] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

[0105] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining a text recognition model, characterized in that: include: Obtaining a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; determining a first target instruction according to a task requirement of the user for the second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; First training data is constructed according to the first target instruction and the first sample image, and model parameters of an initial second text recognition model are adjusted according to the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

2. The method according to claim 1, characterized in that The determining of the first target instruction according to the user's task requirements for the second text recognition task includes: determining initial instructions based on the mission requirements; Perform at least one of rewriting, screening, error correction, and classification on the initial instructions to determine a first target instruction, wherein the rewriting includes synonym replacement and / or grammatical transformation, the screening includes removing at least one of repeated instructions, removing instructions with unclear meanings, and removing instructions with an implementation degree less than a threshold, wherein the implementation degree is used to reflect the degree to which the instructions can be implemented, and the error correction includes correcting spelling errors and / or grammatical errors.

3. The method according to claim 1 or 2, characterized in that The method further comprises: The task requirement is determined according to the user's requirement for the second text recognition task when using the text recognition system, the user's requirement for the second text recognition task in different scenarios, and the user's multiple requirements for multiple second text recognition tasks.

4. The method according to claim 1, wherein The second text recognition model includes a multi-head attention layer, a first additive layer normalization layer, a feedforward network layer, and a second additive layer normalization layer connected in sequence, and adjusting the model parameters of the initial second text recognition model according to the first training data until the model parameters of the second text recognition model meet preset conditions, including: The first training data is input into the initial second text recognition model, and the first parameter of the multi-head attention layer, the second parameter of the first addition and layer normalization layer, the third parameter of the feedforward network layer, and the fourth parameter of the second addition and layer normalization layer are adjusted according to the low-rank decomposition matrix to adjust the model parameters of the initial second text recognition model until the model parameters of the second text recognition model meet preset conditions, thereby obtaining the trained second text recognition model.

5. The method according to claim 1, characterized in that The first text recognition model includes a convolutional neural network, an encoding layer, and a decoding layer.

6. The method according to claim 1 or 5, characterized in that The method further comprises: Obtaining a second target instruction, where the second target instruction is used to reflect the intention of the third text recognition task; Second training data is constructed according to a second target instruction and a second sample image, and model parameters of the pre-trained first text recognition model are updated according to the second training data to obtain an updated first text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the updated first text recognition model.

7. A text recognition method, characterized in that: include: Obtain the image to be recognized; The image to be recognized is processed according to a trained second text recognition model to obtain a text recognition result, wherein the trained second text recognition model is obtained by the method according to any one of claims 1-6.

8. A device for determining a text recognition model, characterized in that: include: A first acquisition module is configured to acquire a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; A first determining module is configured to determine a first target instruction according to a task requirement of a user for a second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; The first training module is configured to construct first training data based on the first target instruction and the first sample image, and adjust the model parameters of the initial second text recognition model based on the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

9. A device for determining a text recognition model, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Obtaining a pre-trained first text recognition model, where the first text recognition model is used to perform a first text recognition task, where the first text recognition task at least includes performing text recognition on image data; determining a first target instruction according to a task requirement of the user for the second text recognition task, wherein the first target instruction is used to reflect the intention of the second text recognition task; First training data is constructed according to the first target instruction and the first sample image, and model parameters of an initial second text recognition model are adjusted according to the first training data until the model parameters of the second text recognition model meet preset conditions, thereby obtaining a trained second text recognition model, wherein the model parameters of the initial second text recognition model are determined based on the model parameters of the first text recognition model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images