Text recognition method and related device

By using word list matching and verification technology in text recognition, the problem of slow text recognition speed is solved, the recognition efficiency and accuracy are improved, and the user experience is improved.

CN117935281BActive Publication Date: 2024-11-05HONOR DEVICE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311808297.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-22
Publication Date
2024-11-05
Estimated Expiration
2043-12-22

AI Technical Summary

Technical Problem

In the prior art, text recognition speed is slow, which affects user experience. It is mainly because large language models require character-by-character reasoning when recognizing, resulting in slower reasoning output.

Method used

By setting the vocabulary list, the characters inferred by using the large language model match the key fields in the vocabulary list, the number of inferences is reduced, the corresponding values ​​of the key fields are directly reasoned, the efficiency of text recognition is improved, and the accuracy is ensured through verification.

Benefits of technology

It improves text recognition speed and accuracy, reduces the number of inferences in large language models, saves inference time, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117935281B_ABST
    Figure CN117935281B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a text recognition method and related devices, which relate to the field of terminal technology. The method includes: obtaining a first text, reasoning the first text based on a target model, a first corresponding relationship, and a second corresponding relationship. In the process of reasoning the first text based on the target model, if the Nth character output by the target model has M suffix characters matching in the first corresponding relationship, the M suffix characters are added after the Nth character, if the first character in the M suffix characters is the same as the first keyword in the second corresponding relationship, the content matching the first keyword is added after the M suffix characters, and the target model reasoning the first text based on the characters output by the target model, the M suffix characters, and the content matching the first keyword, and N and M are both integers greater than or equal to 1. In this way, it is conducive to improving the efficiency of text recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a text recognition method and related devices. Background Art

[0002] Currently, electronic devices can recognize text, obtain information such as time or address in the text, and provide users with corresponding services based on the time or address in the text. For example, in response to a user's screenshot operation, the electronic device can recognize the text on the screenshot image, extract schedule-related information from the text, and create a schedule based on the schedule-related information.

[0003] However, there is a problem of slow text recognition, which affects the user experience. Summary of the invention

[0004] The embodiment of the present application provides a text recognition method, which is applied in the field of terminal technology and is beneficial to improving text recognition efficiency.

[0005] In the first aspect, the embodiment of the present application provides a text recognition method. The method includes: obtaining a first text; reasoning the first text based on a target model, a first correspondence, and a second correspondence, the first correspondence includes a correspondence between a character and a suffix character of the character, and the second correspondence includes a correspondence between a keyword and content matching the keyword in the first text; wherein, in the process of reasoning the first text based on the target model, if the Nth character output by the target model has M suffix characters matching in the first correspondence, the M suffix characters are added after the Nth character, if the first character among the M suffix characters is the same as the first keyword in the second correspondence, the content matching the first keyword is added after the M suffix characters, and the target model reasoning the first text based on the characters output by the target model, the M suffix characters, and the content matching the first keyword, and N and M are both integers greater than or equal to 1.

[0006] In the text recognition method provided by the embodiment of the present application, if the first corresponding relationship includes characters output by the target model, the suffix characters corresponding to the output characters can be added after the characters output by the target model, and these suffix characters do not need to be output character by character, which is beneficial to improving the efficiency of text recognition. In addition, if the keyword included in the second corresponding relationship is the same as one or more characters in the suffix characters, the content matched by the keyword can be added after the suffix characters, and the content matched by the keyword does not need to be output character by character, which is beneficial to further improve the efficiency of text recognition.

[0007] In one possible implementation, if the Nth character output by the target model has M matching suffix characters in the first corresponding relationship, the method also includes: reasoning on the first text based on the target model to obtain (N+M) second characters; adding the M suffix characters after the Nth character, including: if the last M second characters among the (N+M) second characters are the same as the M suffix characters, then adding the M suffix characters after the Nth character.

[0008] The electronic device can verify the M suffix characters, which is helpful to improve the accuracy of text recognition.

[0009] In a possible implementation, the method further includes: if the last M second characters among the (N+M) second characters are different from the M suffix characters, then obtaining the Pth character from the M suffix characters, the Pth character being the first character among the M suffix characters that is different from the characters among the last M second characters; and adding the characters before the Pth character among the M suffix characters to the Nth character. In this way, it is beneficial to improve the text recognition accuracy while improving the text recognition efficiency.

[0010] In a possible implementation, the content matched by the first keyword in the second corresponding relationship includes I characters; if there is a first character in the M suffix characters that is the same as the first keyword in the second corresponding relationship, the method also includes: reasoning the first text based on the target model to obtain (N+M+I) third characters; supplementing the content matched by the first keyword after the M suffix characters, including: if the last I third characters in the (N+M+I) third characters are the same as the content matched by the first keyword, then supplementing the content matched by the first keyword after the M suffix characters. The electronic device can verify the I characters. This is conducive to improving the accuracy of text recognition.

[0011] In a possible implementation, the method further includes: if the last I third characters of the (N+M+I) third characters are different from the content matched by the first keyword, then obtaining the Jth character from the content matched by the first keyword, the Jth character being the first character in the content matched by the first keyword that is different from the characters in the last I third characters; and supplementing the characters before the Jth character in the content matched by the first keyword after the M suffix characters. This is conducive to improving the text recognition efficiency while improving the text recognition accuracy.

[0012] In a possible implementation, before the target model outputs the Nth character, the target model outputs characters one by one.

[0013] In one possible implementation, before obtaining the first text, the method also includes: displaying a chat interface; in response to a screenshot operation, taking a screenshot of the chat interface to obtain a screenshot; obtaining the first text, including: in response to an operation of dragging the screenshot to the first application, or in response to an operation of sharing the screenshot to the first application, obtaining the first text from the screenshot; wherein the characters and suffix characters of the characters included in the first correspondence are high-frequency words used in the first application.

[0014] In this way, the embodiment of the present application can be applicable to the screenshot scenario.

[0015] In a possible implementation, obtaining the first text includes: in response to an operation of invoking a voice assistant, obtaining the user's voice information, and converting the voice information into text information to obtain the first text; the method also includes: identifying the intention of the voice information; if the intention of the voice information is related to the first application, obtaining a first correspondence, and the characters and suffix characters of the characters included in the first correspondence are high-frequency words used in the first application. In this way, the embodiment of the present application can be applied to voice scenarios.

[0016] In a possible implementation, the first application is a calendar application, and the first text is text related to creating a schedule; or, the first application is a gallery application, and the first text is text related to creating a video.

[0017] In a second aspect, an embodiment of the present application provides a text recognition device, which may be an electronic device, or a chip or chip system in an electronic device. The text recognition device may include an acquisition unit and a processing unit. The acquisition unit is used to perform the acquisition step so that the text recognition device implements a text recognition method described in the first aspect or any possible implementation of the first aspect. When the text recognition device is an electronic device, the processing unit may be a processor. The text recognition device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements a text recognition method described in the first aspect or any possible implementation of the first aspect. When the text recognition device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements a text recognition method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or a storage unit in the electronic device located outside the chip (for example, a read-only memory, a random access memory, etc.).

[0018] Exemplarily, an acquisition unit is used to acquire a first text; a processing unit is used to infer the first text based on a target model, a first correspondence relationship, and a second correspondence relationship, the first correspondence relationship including a correspondence relationship between characters and suffix characters of the characters, and the second correspondence relationship including a correspondence relationship between a keyword and content matching the keyword in the first text; wherein, in the process of inferring the first text based on the target model, if the Nth character output by the target model has M suffix characters matching in the first correspondence relationship, the M suffix characters are added after the Nth character, if the first character among the M suffix characters is the same as the first keyword in the second correspondence relationship, the content matching the first keyword is added after the M suffix characters, and the target model infers the first text based on the characters output by the target model, the M suffix characters, and the content matching the first keyword, where N and M are both integers greater than or equal to 1.

[0019] In one possible implementation, if the Nth character output by the target model has M matching suffix characters in the first corresponding relationship, the above-mentioned processing unit is also used to: infer the first text based on the target model to obtain (N+M) second characters; if the last M second characters among the (N+M) second characters are the same as the M suffix characters, then add the M suffix characters after the Nth character.

[0020] In one possible implementation, the processing unit is also used to: if the last M second characters among the (N+M) second characters are different from the M suffix characters, obtain the Pth character from the M suffix characters, the Pth character being the first character among the M suffix characters that is different from the characters among the last M second characters; and add the characters before the Pth character among the M suffix characters to after the Nth character.

[0021] In a possible implementation, the content matched by the first keyword in the second corresponding relationship includes I characters; if there is a first character in the M suffix characters that is the same as the first keyword in the second corresponding relationship, the processing unit is further used to: infer the first text based on the target model to obtain (N+M+I) third characters; if the last I third characters in the (N+M+I) third characters are the same as the content matched by the first keyword, then the content matched by the first keyword is supplemented after the M suffix characters. The electronic device can verify the I characters.

[0022] In a possible implementation, the processing unit is also used for: if the last I third characters among the (N+M+I) third characters are different from the content matched by the first keyword, then obtaining the Jth character from the content matched by the first keyword, where the Jth character is the first character in the content matched by the first keyword that is different from the characters in the last I third characters; and adding the characters before the Jth character in the content matched by the first keyword after the M suffix characters.

[0023] In a possible implementation, before the target model outputs the Nth character, the target model outputs characters one by one.

[0024] In a possible implementation, the text recognition device further includes a display unit. Before acquiring the first text, the display unit is used to: display a chat interface; the processing unit is further used to: in response to a screenshot operation, take a screenshot of the chat interface to obtain a screenshot; the acquisition unit is used to: in response to an operation of dragging the screenshot to the first application, or in response to an operation of sharing the screenshot to the first application, acquire the first text from the screenshot; wherein the characters and suffix characters of the characters included in the first correspondence are high-frequency words used in the first application.

[0025] In one possible implementation, the acquisition unit is used to: in response to an operation of invoking a voice assistant, acquire the user's voice information, and convert the voice information into text information to obtain a first text; the processing unit is also used to: identify the intention of the voice information; if the intention of the voice information is related to the first application, acquire a first corresponding relationship, and the characters and suffix characters of the characters included in the first corresponding relationship are high-frequency words used in the first application.

[0026] In a possible implementation, the first application is a calendar application, and the first text is text related to creating a schedule; or, the first application is a gallery application, and the first text is text related to creating a video.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0029] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0030] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0031] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0032] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0035] Figure 3 A schematic flow chart of a text recognition method provided in an embodiment of the present application;

[0036] Figure 4 A schematic block diagram of a large language model reasoning provided in an embodiment of the present application;

[0037] Figure 5 A schematic block diagram of another large language model reasoning provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram of a schedule creation provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of another schedule creation provided in an embodiment of the present application;

[0040] Figure 8A schematic diagram of another schedule creation provided in an embodiment of the present application;

[0041] Fig. 9 A schematic flow chart of another text recognition method provided in an embodiment of the present application;

[0042] Fig.10 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to clearly describe the technical solution of the embodiment of the present application, the following description is first made:

[0044] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first character and the second character are only used to distinguish different characters, and do not limit their order. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0045] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0046] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0047] Currently, electronic devices can recognize text, obtain information such as time or address in the text, and provide users with corresponding services based on the time or address in the text. For example, in response to a user's screenshot operation, the electronic device can recognize the text on the screenshot image, extract schedule-related information from the text, and create a schedule based on the schedule-related information.

[0048] However, there is a problem of slow text recognition, which affects the user experience.

[0049] The reason why text recognition is slow is that a large language model is generally used for text recognition. The large language model is essentially an autoregressive language model. The autoregressive language model can encode the text from left to right. Since the number of characters required for inference is unknown, one character is output each time the inference is performed. This will lead to slower text recognition.

[0050] To better understand the problem of slow text recognition, the following describes the process of large language model reasoning.

[0051] For example, the text to be recognized by the large language model may be: Meeting at 14:30 tomorrow in the 5th floor conference room, meeting link: http:tmp1.com. The standard output of the large language model for this text is: {"start time":"tomorrow 14:30","location":"5th floor conference room","URL link":"http:tmp1.com","title":"meeting"}.

[0052] The reasoning rules of the large language model may include: 1) encoding the text from left to right; 2) outputting one character each time the reasoning is performed; 3) the input of the next reasoning includes the previous model input and the previous model output; 4) reasoning to the end symbol, for example, or to a preset maximum length, for example, 256 characters.

[0053] Based on this, the first input of the large language model can be: meeting at 14:30 tomorrow in the 5th floor conference room, meeting link: http:tmp1.com. The first output of the large language model can be: {.

[0054] The second input of the large language model can be: There will be a meeting in the conference room on the 5th floor at 14:30 tomorrow, the meeting link is: http:tmp1.com{. The second output of the large language model can be: ".

[0055] The third input of the large language model can be: meeting at 14:30 tomorrow in the conference room on the 5th floor, meeting link: http:tmp1.com{". The third output of the large language model can be: open.

[0056] The fourth input of the large language model can be: meeting at 14:30 tomorrow in the conference room on the 5th floor, meeting link: http:tmp1.com{“start. The fourth output of the large language model can be: start.

[0057] The fifth input to the large language model can be: There will be a meeting at 2:30 PM tomorrow in the meeting room on the 5th floor. Meeting link: http: / / tmp1.com {"Start. The fifth output of the large language model can be: Time.

[0058] The sixth input to the large language model can be: There will be a meeting at 2:30 PM tomorrow in the meeting room on the 5th floor. Meeting link: http: / / tmp1.com {"Start time. The sixth output of the large language model can be: Time.

[0059] And so on, through the large language model output: {"Start time": "Tomorrow 14:30", "Location": "Meeting room on the 5th floor", "URL link": "http: / / tmp1.com", "Title": "Have a meeting"}.

[0060] It can be understood that the large language model can generate the above results by word-by-word reasoning, that is, the first word generates: {, the second word generates: ", the third word generates: H, the fourth word generates: a, …, the last word generates:}.

[0061] In this way, each time a character is output by reasoning, the number of characters to be output determines the number of reasoning times, resulting in a slow reasoning output and thus a slow text recognition speed.

[0062] In view of this, the embodiments of the present application provide a text recognition method and related device, which set a vocabulary table. The structure of the strings in the vocabulary table is a JSON structure. When the characters inferred by the large language model are the same as the key fields in the strings, the values corresponding to the key fields can also be used as the output of the large language model. In this way, there is no need for the large language model to infer the values corresponding to the key fields, reducing the number of reasoning times, which is beneficial to improving the reasoning speed and thus the text recognition speed. Among them, the vocabulary table can also be called a dictionary, and the embodiments of the present application do not limit the name.

[0063] Exemplarily, in the above example, the application scenario is a schedule extraction scenario. In this scenario, the strings in the vocabulary table can include {"Ti": "tle": "", "St": "art time": "", "En": "d time": "", "Re": "mind time": "", "Re": "peat cycle": "", "U": "RL link": "", "Lo": "cation": ""}. The structures of these strings are all JSON structures. For example, the key field is: Ti, and the value corresponding to the key field is: tle": "". The key field is: St, and the value corresponding to the key field is: art time": "".

[0064] The JSON structure can be understood as having a matching mapping. For example, "Ti" corresponds to "tle": "", "St" matches "art time": "", and "U" corresponds to "RL link": "".

[0065] In the above example, if the text: Meeting at 14:30 tomorrow in the conference room on the 5th floor, meeting link: http:tmp1.com is recognized based on the method provided in the embodiment of the present application, then when the large language model infers the third word, the third output of the large language model is: open, and in the vocabulary, "start time":"" corresponds to "open", then "start time":"" can be used as the third output of the large language model. In this way, the fourth input of the large language model can be: Meeting at 14:30 tomorrow in the conference room on the 5th floor, meeting link: http:tmp1.com{"start time":"". The fourth output of the large language model can be: Ming.

[0066] From this, we can see that the "start time":" does not need the large language model to reason word by word, and can skip the reasoning process of these 6 characters and directly reason the word "Ming". Similarly, when the large language model infers "di", the "point":"" does not need the large language model to reason word by word, and can skip the reasoning process of these 4 characters and directly reason the word "5". When the large language model infers "U", the "RL link":"" does not need the large language model to reason word by word, and can directly reason the word "h".

[0067] In this way, multiple characters can be obtained at one time through the vocabulary, without the need to infer character by character, which reduces the number of inferences of the large language model, helps to improve the inference speed, and then improves the text recognition speed.

[0068] The method provided in the embodiment of the present application can be applied to electronic devices, and can also be applied to the cloud, such as a cloud server. The embodiment of the present application does not limit this. If an electronic device executes the method provided in the embodiment of the present application, it is beneficial to improve the reasoning performance of the electronic device. If the method provided in the embodiment of the present application is executed in the cloud, it is beneficial to save the power consumption of the electronic device. If the method provided in the embodiment of the present application is executed in the cloud, the electronic device can send the text to be recognized to the remote end, and the remote end can recognize the text based on a static vocabulary and a large language model, and send the recognition result to the electronic device.

[0069] The electronic devices involved in the embodiments of the present application may include handheld devices, vehicle-mounted devices, etc. with text recognition functions. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (PLANs), and wireless terminals in smart cities. The embodiments of the present application do not limit this.

[0070] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0071] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0072] To facilitate understanding, the hardware structure of the electronic device according to the embodiment of the present application is introduced below.

[0073] For example, Figure 1 The hardware structure diagram of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a display screen 194, etc.

[0074] Optionally, the above-mentioned sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0075] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0076] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The layered architecture can adopt an Android system, an Apple (IOS) system, or other operating systems, which are not limited in the embodiments of the present application. The following takes the Android system of the layered architecture as an example to exemplify the software architecture of the electronic device provided in the embodiments of the present application.

[0077] Figure 2 A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application. Figure 2 As shown, the layered architecture divides the software system of the electronic device into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom, namely, the application layer (applications), the application framework layer (application framework), the hardware abstraction layer (hardware abstraction layer, HAL), the kernel layer (kernel), and the hardware layer. The application layer can include a series of application packages, and the application layer runs the application by calling the application programming interface (application programming interface, API) provided by the application framework layer. Figure 2 As shown, the application package may include applications such as chat, calendar, gallery, and video.

[0078] The application framework layer provides API and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 2 As shown, the application framework layer may include a text recognition module, a view system, and a content provider, etc.

[0079] Among them, the text recognition module can execute the method provided by the embodiment of the present application. When there is a need for text recognition, the text recognition module can call the large language model to recognize the text. When the inferred text can be found in the vocabulary, the text in the vocabulary can be used as the inference result of the large language model to continue to infer other text.

[0080] It should be noted that the text recognition module can be deployed in the application framework layer or in other layers except the application framework layer, and the embodiments of the present application do not limit this. The software architecture of the electronic device is generally deployed in the application processor (AP) chip of the electronic device, and the text recognition module can be deployed in the AP chip or in other chips in the electronic device, and the embodiments of the present application do not limit this.

[0081] The purpose of the HAL layer is to abstract the hardware and provide a unified interface for querying hardware devices for upper-layer applications, or to provide data storage services for upper-layer applications. Figure 2 As shown, the HAL layer may include a display driver module and a sensor hardware abstraction (sensor hidl).

[0082] The kernel layer is the layer between hardware and software. Figure 2 As shown, the kernel layer may include one or more of the following: sensor driver, display driver, etc. In the embodiment of the present application, the sensor driver includes a sensor subsystem framework and a sensor subsystem driver, which are located in the coprocessor (SCP).

[0083] The hardware layer may include hardware such as microphones, cameras, and displays.

[0084] It should be understood that in some embodiments, the layers implementing the same function may be called by other names, or the layers that can implement the functions of multiple layers may be regarded as one layer, or the layers that can implement the functions of multiple layers may be divided into multiple layers. The embodiments of the present application are not limited to this.

[0085] Combined with the above Figure 1 and Figure 2 An electronic device according to an embodiment of the present application has been introduced. The method provided by an embodiment of the present application is introduced below by taking an electronic device executing the method provided by an embodiment of the present application as an example.

[0086] Before introducing the method provided in the embodiment of the present application, the vocabulary used in the embodiment of the present application is first introduced.

[0087] The vocabulary used in the embodiments of the present application may include a static vocabulary and a dynamic vocabulary. The structure of the string in the static vocabulary and the dynamic vocabulary may be a json structure, and the static vocabulary and the dynamic vocabulary both include a corresponding relationship. The corresponding relationship in the static vocabulary may be a pre-made fixed corresponding relationship, which will not change based on the change of the text to be recognized, and the corresponding relationship in the dynamic vocabulary may be generated based on the text to be recognized, which may change based on the change of the text to be recognized.

[0088] The following is an introduction to the method provided in the embodiments of the present application.

[0089] Figure 3 FIG. 1 is a schematic flow chart of a text recognition method provided by an embodiment of the present application. Figure 3 As shown, the method may include the following steps:

[0090] S301: When there is a need for text recognition, load a large language model and a static vocabulary.

[0091] The large language model may also be referred to as a large model, which is not limited in the embodiments of the present application.

[0092] Electronic devices may have different text recognition requirements in different scenarios, and the loaded static word lists may be different in different scenarios.

[0093] Exemplarily, in a schedule extraction scenario, the static vocabulary may include {"title":"","start":"start time":"","end":"end time":"","reminder":"wake-up time":"","repetition cycle":"","URL":"RL link":"","place":"point":""}. In a video production scenario, the static vocabulary may include {"time":"time","place":"point":"","people":"things":"","task":""}.

[0094] Take the schedule extraction scenario as an example. The electronic device can provide the user with the function of creating a schedule through a screenshot operation. When the electronic device detects the user's screenshot operation, in response to the user's screenshot operation, the large language model and static vocabulary are loaded to recognize the text on the screenshot image, extract the schedule-related information in the text, and create a schedule based on the schedule-related information.

[0095] S302: Obtain the text to be recognized.

[0096] The text to be recognized may be input by a user or recognized by an electronic device, and this embodiment of the present application does not limit this.

[0097] Exemplarily, the text to be recognized may be as shown in the above example: a meeting will be held in the conference room on the 5th floor at 14:30 tomorrow, meeting link: http:tmp1.com, or it may be: help me make a video of my son's 3-year-old birthday in a restaurant, etc. The embodiments of the present application are not limited to this.

[0098] In the schedule extraction scenario, the electronic device can recognize the text on the screenshot image to obtain text, for example, a meeting will be held at 14:30 tomorrow in the conference room on the 5th floor, and the meeting link is: http:tmp1.com. It is understandable that the image also includes other text, which the electronic device can also recognize on the screenshot image, but it is not explained here because it is related to the schedule.

[0099] S303: According to the key field in the dynamic vocabulary, extract the value corresponding to the key field from the text.

[0100] The key field in the dynamic vocabulary is preset, and the value corresponding to the key field needs to be identified from the text. In different scenarios, the dynamic vocabulary can be different.

[0101] For example, in a schedule extraction scenario, the key field in the dynamic vocabulary may include {"time", "URL link", "location"}. In a video production scenario, the static vocabulary may include {"time", "location", "person"}.

[0102] In the schedule extraction scenario, if the text to be recognized is: Meeting at 14:30 tomorrow in the 5th floor conference room, meeting link: http:tmp1.com, the electronic device can extract the value corresponding to the key field from the text according to the key field in the dynamic vocabulary. If the key field in the dynamic vocabulary can include {"time", "URL link", "location"}, the value corresponding to "time" can be extracted as: tomorrow 14:30, the value corresponding to "URL link" is: http:tmp1.com, and the value corresponding to "location" is: 5th floor conference room.

[0103] S304: Update the dynamic vocabulary.

[0104] The electronic device can add the value corresponding to the key field to the dynamic vocabulary to update the dynamic vocabulary.

[0105] For example, in the schedule extraction scenario, the electronic device extracts the value corresponding to "time": tomorrow 14:30, the value corresponding to "URL link": http:tmp1.com, and the value corresponding to "location": 5th floor conference room, then these information can be added to the dynamic vocabulary to update the dynamic vocabulary. The updated dynamic vocabulary can be {"time": "tomorrow 14:30", "url link": "http:tmp1.com", "location": "5th floor conference room"}.

[0106] S305: Infer the text through a large language model to obtain a token.

[0107] Token is used to represent the characters inferred by the large language model. The inference rules of the large language model may include: 1) Encode the text from left to right; 2) Output one character each time inference; 3) The input of the next inference includes the previous model input and the previous model output. 4) Inference ends with the end symbol, for example, or inference ends with a preset maximum length, for example, 256 characters.

[0108] Since the large language model infers one character at a time, a token can be obtained by inferring the text through the large language model.

[0109] S306: Determine whether the token is in the static vocabulary or the dynamic vocabulary.

[0110] If the token is in the static vocabulary or the dynamic vocabulary, the electronic device can use the characters in the static vocabulary or the dynamic vocabulary as the result of the large language model inference, and can execute S307. If the token is not in the static vocabulary or the dynamic vocabulary, the electronic device can continue to infer the next token through the large language model, and can execute S312.

[0111] In the schedule extraction scenario, the text to be recognized is: Meeting at 14:30 tomorrow in the conference room on the 5th floor, meeting link: http:tmp1.com, then the first token inferred by the large speech model is: {, this token is not in the static vocabulary or dynamic vocabulary, the electronic device can continue to infer the next token through the large language model, that is, it can execute S312. When the large speech model infers "open", this token is in the static vocabulary, the electronic device can use the "start time" of the static vocabulary:"" as the result inferred by the large language model, that is, it can execute S307.

[0112] In some examples, the electronic device may first determine whether the token is in the static vocabulary, and if not, then determine whether the token is in the dynamic vocabulary. This is helpful to improve the accuracy of text recognition.

[0113] S307: If the token is in the static vocabulary or the dynamic vocabulary, the electronic device can obtain n tokens through the static vocabulary or the dynamic vocabulary, where n is an integer greater than or equal to 1.

[0114] It is understandable that these n tokens are the values ​​corresponding to the key field. For example, when the large speech model infers "on", the electronic device can obtain the value "start time" corresponding to "on" through a static word list or a dynamic word list, and obtain 6 tokens.

[0115] S308: Verify the n tokens through a large language model.

[0116] Since the large speech model determines the number of characters to be inferred (the characters output by the large speech model and the n characters), the large speech model can infer these multiple characters at the same time. If the n tokens are verified, the large speech model can infer the n characters at the same time and determine whether the inferred n characters are the same as the n tokens. If the n characters are the same as the n tokens, it means that the n tokens obtained through the static vocabulary or dynamic vocabulary are the characters required to be inferred, and the verification is passed. If the n characters are different from the n tokens, it means that there are wrong characters in the n tokens obtained through the static vocabulary or dynamic vocabulary, and the verification fails.

[0117] In this way, verifying the n tokens obtained through the static word list or the dynamic word list is conducive to improving the accuracy of text recognition. In addition, although the embodiment of the present application adds a verification process, the embodiment of the present application verifies multiple tokens in parallel, which can still save time and improve text recognition efficiency compared to reasoning one by one.

[0118] S309: Determine whether the verification is successful.

[0119] S310: If the verification is successful, the electronic device can obtain n tokens.

[0120] After the electronic device obtains n tokens, it can determine whether there are one or more characters in the static vocabulary or dynamic vocabulary among the n tokens. If there are one or more characters in the static vocabulary or dynamic vocabulary among the n tokens, the electronic device can use the characters in the static vocabulary or dynamic vocabulary as the result of inference by the large language model, that is, the electronic device can add these n tokens after the token output by the large speech model. In this way, the reasoning time can be further saved, which is conducive to improving the reasoning speed.

[0121] Exemplarily, if n tokens are "starting time":"", and "time" exists in the dynamic vocabulary among these n tokens, the electronic device can add the value "tomorrow 14:30" corresponding to "time" after "starting time":"". The electronic device can determine whether there are one or more characters in "tomorrow 14:30" in the static vocabulary or the dynamic vocabulary. If there are one or more characters in "tomorrow 14:30" in the static vocabulary or the dynamic vocabulary, the electronic device can use the characters in the static vocabulary or the dynamic vocabulary as the result inferred by the large language model.

[0122] If there is no character in the static vocabulary or the dynamic vocabulary among the n tokens, the electronic device can infer the next token through the large language model.

[0123] For example, Figure 4 A schematic diagram of a large language model inference is shown. Figure 4 As shown, the tokens inferred by the large language model include 1026, 1244, 307, 1744 and 284. These tokens can be inferred word by word by the large language model. These numbers are used to indicate the position of the token in the preset vocabulary, which can include many words, such as 100,000 words. 1026 can be used to indicate that the first token is the 1026th word in the preset vocabulary, and 1244 can be used to indicate that the second token is the 1244th word in the preset vocabulary. Other similarities will not be described in detail here.

[0124] Five tokens are obtained from the static vocabulary through 284, and the five tokens include 466, 428, 11, 475, and 340. Then the result of this reasoning may include 1026, 1244, 307, 1744, 284, 466, 428, 11, 475, and 340. The electronic device can verify these five tokens through the large language model, that is, after the inferred tokens, five tokens are inferred in parallel. If the inferred tokens include 466, 428, 11, 475, and 340, which are the same as the five tokens obtained from the static vocabulary, the verification is passed.

[0125] If the verification is successful, the electronic device can infer the next token based on the result of this inference, for example, Figure 4 As shown in the figure, the next token is inferred to be 338 through the large language model.

[0126] S311. If the verification fails, the electronic device can obtain the first verification error token.

[0127] If the verification fails, it means that there is at least one token error. If there is a token error, the first token with verification error is the wrong token. If there are multiple token errors, the first token with verification error is the first wrong token from left to right.

[0128] S312. The electronic device may start from the first token with incorrect verification and infer the next token through the large language model.

[0129] The electronic device can start from the first token with incorrect verification, delete the subsequent tokens, retain the token before the first token with incorrect verification, and based on this, infer the next token through the large language model. In other words, the electronic device can add the token before the first token with incorrect verification among the n tokens after the token output by the large speech model, and based on this, infer the next token through the large language model.

[0130] For example, Figure 5 A schematic diagram of a large language model inference is shown. Figure 5As shown in the figure, the tokens inferred by the large language model include 1026, 1244, 307, 1744, and 284. Five tokens are obtained from the static vocabulary through 284, and the five tokens include 466, 428, 11, 475, and 340. The result of this inference may include 1026, 1244, 307, 1744, 284, 466, 428, 11, 475, and 340.

[0131] The electronic device can verify these five tokens through a large language model, that is, after the inferred token, five more tokens are inferred in parallel. If the inferred tokens include 466, 428, 287, 475, and 340, and one token is different from the five tokens obtained from the static vocabulary, the verification fails.

[0132] If the verification fails, the electronic device can obtain the first verification error token, that is, 11 is different from 287, then the tokens before 11 are retained, and the tokens after 11 and 11 are deleted, that is, the result of this reasoning is updated, and the updated reasoning results include 1026, 1244, 307, 1744, 284, 466 and 428. The electronic device can infer the next token based on the result of this reasoning, for example, Figure 5 As shown in the figure, the next token is inferred to be 287 through the large language model.

[0133] S313: Determine whether the next token is a stop character.

[0134] The stop character may be / s or / n, etc., which is not limited in the present embodiment. The stop character may also be called an end character, which is not limited in the present embodiment.

[0135] If the next token is a stop character, it means that the text recognition is terminated and the electronic device can stop the recognition. If the next token is not a stop character, it means that the reasoning needs to be continued and the electronic device can determine whether the token is in the static word list or the dynamic word list, that is, execute S306.

[0136] S314: If the next token is a stop character, output the recognition result.

[0137] Depend on Figure 4 It can be seen that the electronic device can infer the next token through the large language model in any of the following situations:

[0138] The tokens obtained by the electronic device through the large language model are neither in the static vocabulary nor in the dynamic vocabulary;

[0139] The token obtained by the electronic device through the large language model is in the static vocabulary or dynamic vocabulary, but the token obtained through the static vocabulary or dynamic vocabulary has not been verified; or,

[0140] The token obtained by the electronic device through the large language model is in the static vocabulary or the dynamic vocabulary, and the token obtained through the static vocabulary or the dynamic vocabulary is verified, but the token obtained through the static vocabulary or the dynamic vocabulary is not in the static vocabulary or the dynamic vocabulary.

[0141] The text recognition method provided in the embodiment of the present application matches the tokens inferred by the large language model through a static vocabulary and / or a dynamic vocabulary to obtain n tokens, which is conducive to reducing the number of inferences and improving the text recognition speed. In addition, the embodiment of the present application verifies the matched tokens to improve the accuracy of text recognition. In addition, in the case where the verification of the matched token fails, the correct token is retained, the wrong token is deleted, and the wrong token is inferred, which is conducive to improving the text recognition speed while ensuring the accuracy of text recognition.

[0142] The above-mentioned large language model can be trained through the following steps: the electronic device can input an instruction to the source model and give a specific answer, so that the source model recognizes the text through the instruction, so that the output is close to or the same as the specific answer.

[0143] Exemplarily, the instructions of the source model may be: The following is the content input by the user, which may include several events, each event contains fields such as <"title", "start time", "end time", "recurrence", "location", "url link", "participants">, and each field existing in the event is output as a line in json format, without additional response.

[0144] The source model may recognize texts based on the instruction: At 3 pm on Wednesday, I will go to Xinhua Bookstore to apply for a card and buy some books, which will take a while. On Sunday, I will go to a car repair shop to wash the car and clean the interior, which is very important. On Saturday, I will clean the house, including the refrigerator, TV, and computer, so I may be a little busy.

[0145] The standard output of this text is {"Title":"Apply for a card and buy some books at the same time","Start Time":"Wednesday 03:00pm","Location":"Xinhua Bookstore"}, {"Title":"Cleaning the house","Start Time":"Saturday","Location":"Home"}, {"Title":"Car washing and interior cleaning","Start Time":"Sunday","Location":"Auto repair shop"}.

[0146] The source model recognizes the text through instructions, and if the output is close to or the same as the standard output, it converges and obtains a large language model.

[0147] The method provided by the embodiment of the present application is described above, and the application scenarios applicable to the embodiment of the present application are introduced below.

[0148] In some implementations, Figure 6 A schematic diagram of schedule creation is shown. Figure 6 As shown, the electronic device is a mobile phone. Figure 6 As shown in interface a in the figure, the mobile phone displays a chat interface. The chat object displayed in the chat interface is "partner". The chat content can be summarized as: the partner arranges the user to discuss business at 206, Section 10, East Main Building of the school at 10:20 on Friday. In response to the screenshot operation, the mobile phone can take a screenshot of the chat interface to obtain a screenshot, which can be displayed in the form of a window on the chat interface, such as Figure 6 As shown in the b interface. Figure 6 In the b interface, the mobile phone also displays a sharing control. In response to the user triggering the sharing control, the mobile phone can display Figure 6 In the c interface. Figure 6 As shown in the c interface in the figure, the mobile phone supports users to share screenshots to applications such as calendar, gallery or memo. In response to the user's operation of sharing the screenshot to the calendar application, the text in the screenshot is obtained: I will go to school on Friday, let's chat; OK; let's meet at 206, Section 10, East Main Building at 10:20; no problem. The mobile phone can be used through the above Figure 3 The method shown recognizes the text and outputs {"title":"Exchange business","start time":"Friday 10:20","location":"Room 206, Area 10, East Main Building of the school"}.

[0149] In other implementations, Figure 7 A schematic diagram of another schedule creation is shown. Figure 7 As shown, the electronic device is a mobile phone. Figure 7 As shown in interface a in the figure, the mobile phone displays a chat interface. The chat object displayed in the chat interface is "partner". The chat content can be summarized as: the partner arranges the user to discuss business at 206, Section 10, East Main Building of the school at 10:20 on Friday. In response to the screenshot operation, the mobile phone can take a screenshot of the chat interface to obtain a screenshot, which can be displayed in the form of a window on the chat interface, such as Figure 7 As shown in the b interface. Figure 7 In the b interface, the mobile phone can support users to drag and drop screenshots. During the drag and drop screenshot process, the sharing control may not be displayed. In response to the user dragging the screenshot to the right side of the mobile phone, a sidebar is displayed, such as Figure 7 As shown in the c interface. Figure 7In the c interface in the figure, the sidebar includes applications such as calendar, gallery, and memo. The mobile phone supports users to drag and drop screenshots to the calendar, gallery, or memo. In response to the user dragging the screenshot to the calendar application, the text in the screenshot is obtained: I will go to school on Friday, let's chat; OK; let's meet at 206, Section 10, East Main Building at 10:20; no problem. The mobile phone can use the above Figure 3 The method shown recognizes the text and outputs {"title":"Exchange business","start time":"Friday 10:20","location":"Room 206, Area 10, East Main Building of the school"}.

[0150] Among them, in the above Figure 7 In the c interface, the sidebar can be displayed in response to the user dragging the screenshot to the right side of the mobile phone, or it can be displayed in response to the user long pressing the screenshot. The embodiment of the present application is not limited to this.

[0151] In some further implementations, Figure 8 A schematic diagram of another schedule creation is shown. Figure 8 As shown, the electronic device is a mobile phone. When the mobile phone displays the main interface, in response to the user's operation of invoking the voice assistant, the user's voice information is obtained and converted into text information to obtain the text: I have a meeting at 3 o'clock tomorrow. At the same time, the mobile phone recognizes the intention of the voice information and determines that a schedule needs to be established. It is related to the calendar application, and can be based on the above Figure 3 The method shown recognizes the text and outputs {"title":"Meeting","start time":"tomorrow 3:00"}. At the same time, the time on the mobile phone can be obtained: Tuesday, November 1, and tomorrow is determined to be Wednesday, November 2. Based on this information, a schedule can be created and saved to the calendar. At the same time, the text "Schedule has been added to the calendar" and schedule-related information are displayed. Figure 8 As shown, the mobile phone has created a meeting in the calendar application with the title "Meeting" and the meeting time is 03:00-04:00 on Wednesday, November 2. It can be understood that the meeting duration is the default.

[0152] Above Figures 6 to 8 All of them describe schedule creation scenarios, or schedule extraction scenarios. The embodiments of the present application are also applicable to video creation scenarios.

[0153] For example, in the above Figure 8 In the scenario of using a voice assistant shown in the figure, if the text obtained based on the user's voice information is: Help me make a video of my son's 3-year-old birthday in a restaurant, the mobile phone can recognize the intention of the voice information and determine that a video needs to be made. It is related to the gallery application, and then based on the above Figure 3The method shown recognizes the text and outputs {"title":"birthday","location":"restaurant","character":"son"}. Based on this information, photos that meet the conditions can be found in the gallery for video production. After the production is completed, the text can be displayed: the video has been produced.

[0154] The method provided in the embodiment of the present application is introduced above in combination with the usage scenario, and the method provided in the embodiment of the present application will be introduced below in combination with the step process.

[0155] For example, Fig. 9 FIG. 1 is a schematic flow chart of a text recognition method provided by an embodiment of the present application. Fig. 9 As shown, the method may include the following steps:

[0156] S901. Obtain a first text.

[0157] The first text may also be described as text to be recognized, or text to be recognized, which is not limited in this embodiment of the present application.

[0158] In the above Figure 6 or Figure 7 In the example shown, the first text is taken from the screenshot. Figure 8 In the example shown, the first text is converted based on the user's voice information.

[0159] S902, reasoning the first text based on the target model, the first correspondence, and the second correspondence, wherein the first correspondence includes the correspondence between characters and suffix characters of the characters, and the second correspondence includes the correspondence between the keyword and the content matching the keyword in the first text. In the process of reasoning the first text based on the target model, if the Nth character output by the target model has M suffix characters matching in the first correspondence, the M suffix characters are added after the Nth character, and if the first character among the M suffix characters is the same as the first keyword in the second correspondence, the content matching the first keyword is added after the M suffix characters. The target model reasoning the first text based on the characters output by the target model, the M suffix characters, and the content matching the first keyword, where N and M are both integers greater than or equal to 1.

[0160] The target model can refer to the above-mentioned large speech model. The first correspondence can refer to the above-mentioned static vocabulary, and the second correspondence can refer to the above-mentioned dynamic vocabulary. The first correspondence and the second correspondence can be key-value pairs, or other correspondences, which are not limited in the embodiments of the present application. If the first correspondence is a key-value pair, the characters in the first correspondence are used to represent the key field, and the suffix characters of the characters are used to represent the value corresponding to the key field. If the second correspondence is a key-value pair, the keywords in the second correspondence are used to represent the key field, and the content matching the keywords is used to represent the value corresponding to the key field.

[0161] The M suffix characters may include M characters that match the Nth character output by the target model in the first correspondence. The content of the M suffix characters matching the first keyword can refer to the above Figure 3 The first character may include one or more characters, which is not limited in this embodiment of the present application.

[0162] Before the target model outputs the Nth character, the characters output by the target model may be output character by character, or may include characters obtained by the first corresponding relationship and / or the second corresponding relationship, which is not limited in the embodiment of the present application.

[0163] In the text recognition method provided by the embodiment of the present application, if the first corresponding relationship includes characters output by the target model, the suffix characters corresponding to the output characters can be added after the characters output by the target model, and these suffix characters do not need to be output character by character, which is beneficial to improving the efficiency of text recognition. In addition, if the keyword included in the second corresponding relationship is the same as one or more characters in the suffix characters, the content matched by the keyword can be added after the suffix characters, and the content matched by the keyword does not need to be output character by character, which is beneficial to further improve the efficiency of text recognition.

[0164] Optionally, if the Nth character output by the target model has M matching suffix characters in the first corresponding relationship, the method also includes: reasoning on the first text based on the target model to obtain (N+M) second characters; adding the M suffix characters after the Nth character, including: if the last M second characters among the (N+M) second characters are the same as the M suffix characters, then adding the M suffix characters after the Nth character.

[0165] The electronic device can verify M suffix characters. When verifying M suffix characters, since the number of characters to be inferred is known, (N+M) characters can be inferred in parallel to obtain (N+M) second characters. If the last M second characters of the (N+M) second characters are the same as the M suffix characters, it can be said that the verification is passed, and the electronic device can add the M suffix characters after the Nth character. This is conducive to improving the accuracy of text recognition.

[0166] Optionally, the method also includes: if the last M second characters among the (N+M) second characters are different from the M suffix characters, then obtaining the Pth character from the M suffix characters, the Pth character being the first character among the M suffix characters that is different from the characters among the last M second characters; and adding the characters before the Pth character among the M suffix characters to after the Nth character.

[0167] If the last M second characters of the (N+M) second characters are different from the M suffix characters, it means that the verification fails, and the first wrong character, that is, the Pth character, can be obtained. The electronic device can add the characters before the first wrong character after the Nth character. In this way, it is beneficial to improve the text recognition efficiency and the text recognition accuracy at the same time.

[0168] Optionally, the content matched by the first keyword in the second corresponding relationship includes I characters; if there is a first character among the M suffix characters that is the same as the first keyword in the second corresponding relationship, the method also includes: reasoning on the first text based on the target model to obtain (N+M+I) third characters; supplementing the content matched by the first keyword after the M suffix characters, including: if the last I third characters among the (N+M+I) third characters are the same as the content matched by the first keyword, then supplementing the content matched by the first keyword after the M suffix characters.

[0169] The electronic device can verify I characters. When verifying I characters, since the number of characters to be inferred is known, (N+M+I) characters can be inferred in parallel to obtain (N+M+I) third characters. If the last I third characters of the (N+M+I) third characters are the same as the content matched by the first keyword, it can be said that the verification is passed, and the electronic device can add I characters after the M suffix characters. This is conducive to improving the accuracy of text recognition.

[0170] Optionally, the method also includes: if the last I third characters among the (N+M+I) third characters are different from the content matched by the first keyword, then obtaining the Jth character from the content matched by the first keyword, the Jth character being the first character in the content matched by the first keyword that is different from the characters in the last I third characters; and adding the characters before the Jth character in the content matched by the first keyword after the M suffix characters.

[0171] If the last I third characters among the (N+M+I) third characters are different from the content matched by the first keyword, it can be said that the verification fails, and the electronic device can obtain the first wrong character, that is, the Jth character, and the electronic device can add the characters before the first wrong character after the Nth character. In this way, it is beneficial to improve the text recognition efficiency and the text recognition accuracy at the same time.

[0172] Optionally, before the target model outputs the Nth character, the target model outputs characters one by one.

[0173] Optionally, before obtaining the first text, the method also includes: displaying a chat interface; in response to a screenshot operation, taking a screenshot of the chat interface to obtain a screenshot; obtaining the first text, including: in response to an operation of dragging the screenshot to the first application, or in response to an operation of sharing the screenshot to the first application, obtaining the first text from the screenshot; wherein the characters and suffix characters of the characters included in the first correspondence are high-frequency words used in the first application.

[0174] The first application may be an application such as a calendar, a gallery, or a memo, and the present application embodiment does not limit this. If the first application is a calendar, the scenario of sharing the screenshot to the first application can refer to the above Figure 6 , drag the screenshot to the first application scenario can refer to the above Figure 7 In this way, the embodiment of the present application can be applicable to the screenshot scenario.

[0175] Optionally, obtaining the first text includes: in response to an operation of invoking a voice assistant, obtaining the user's voice information, and converting the voice information into text information to obtain the first text; the method also includes: identifying the intention of the voice information; if the intention of the voice information is related to the first application, obtaining a first corresponding relationship, and the characters and suffix characters of the characters included in the first corresponding relationship are high-frequency words used in the first application.

[0176] If the first application is a calendar, the scenario of creating a schedule based on the user's voice information can refer to the above Figure 8 In this way, the embodiment of the present application can be applicable to voice scenarios.

[0177] Optionally, the first application is a calendar application, and the first text is text related to creating a schedule; or, the first application is a gallery application, and the first text is text related to creating a video.

[0178] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0180] The text recognition method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above text recognition method.

[0181] Fig.10 This is a schematic diagram of the structure of a chip provided in an embodiment of the present application. Fig.10 As shown, the chip 100 includes one or more (including two) processors 1001 , a communication line 1002 , a communication interface 1003 and a memory 1004 .

[0182] In some implementations, the memory 1004 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0183] The text recognition method described in the above embodiment of the present application can be applied to the processor 1001, or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above text recognition method can be completed by an integrated logic circuit of hardware or software instructions in the processor 1001. The above processor 1001 can be a general processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components. The processor 1001 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application.

[0184] The steps of the text recognition method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 1004, and the processor 1001 reads the information in the memory 1004 and completes the steps of the above method in combination with its hardware.

[0185] The processor 1001 , the memory 1004 , and the communication interface 1003 may communicate with each other via the communication line 1002 .

[0186] In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.

[0187] The text recognition method provided in the embodiment of the present application can be applied to electronic devices with processing functions. The electronic devices include terminal devices, and the specific device form of the terminal device can refer to the above related descriptions, which will not be repeated here.

[0188] An embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.

[0189] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0190] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0191] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0192] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0193] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0194] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A text recognition method, It is characterized in that include: Get the first text; Reasoning the first text based on the target model, a first correspondence relationship, and a second correspondence relationship, wherein the first correspondence relationship includes a correspondence relationship between a character and a suffix character of the character, and the second correspondence relationship includes a correspondence relationship between a keyword and content matching the keyword in the first text; In which, in the process of reasoning the first text based on the target model, if the Nth character output by the target model has M matching suffix characters in the first corresponding relationship, the M suffix characters are added after the Nth character, and if there is a first character among the M suffix characters that is the same as the first keyword in the second corresponding relationship, the content matching the first keyword is added after the M suffix characters. The target model infers the first text based on the characters output by the target model, the M suffix characters and the content matching the first keyword, and N and M are both integers greater than or equal to 1.

2. The method according to claim 1, It is characterized in that If the Nth character output by the target model has M matching suffix characters in the first corresponding relationship, the method further includes: Reasoning the first text based on the target model to obtain (N+M) second characters; The step of adding the M suffix characters to the Nth character includes: If the last M second characters among the (N+M) second characters are the same as the M suffix characters, the M suffix characters are added after the Nth character.

3. The method according to claim 2, It is characterized in that The method further comprises: If the last M second characters of the (N+M) second characters are different from the M suffix characters, then a P-th character is obtained from the M suffix characters, and the P-th character is the first character of the M suffix characters that is different from a character in the last M second characters; The characters before the P-th character among the M suffix characters are added after the N-th character.

4. The method according to claim 1 or 2, It is characterized in that In the second corresponding relationship, the content matched by the first keyword includes 1 character; If a first character in the M suffix characters is the same as the first keyword in the second corresponding relationship, the method further includes: Reasoning the first text based on the target model to obtain (N+M+I) third characters; The step of adding the content matched by the first keyword after the M suffix characters includes: If the last I third characters among the (N+M+I) third characters are the same as the content matched by the first keyword, the content matched by the first keyword is supplemented after the M suffix characters.

5. The method according to claim 4, It is characterized in that The method further comprises: If the last I third characters of the (N+M+I) third characters are different from the content matched by the first keyword, then obtain the Jth character from the content matched by the first keyword, and the Jth character is the first character in the content matched by the first keyword that is different from the characters in the last I third characters; The characters before the Jth character in the content matched by the first keyword are added after the M suffix characters.

6. The method according to any one of claims 1 to 5, It is characterized in that Before the target model outputs the Nth character, the target model outputs characters one by one.

7. The method according to any one of claims 1 to 6, It is characterized in that Before obtaining the first text, the method further includes: Display the chat interface; In response to a screenshot operation, taking a screenshot of the chat interface to obtain a screenshot; The obtaining of the first text comprises: In response to an operation of dragging the screenshot to a first application, or in response to an operation of sharing the screenshot to the first application, acquiring the first text from the screenshot; The characters and suffix characters of the characters included in the first corresponding relationship are high-frequency words used in the first application.

8. The method according to any one of claims 1 to 6, It is characterized in that The obtaining of the first text comprises: In response to the operation of invoking the voice assistant, obtaining voice information of the user, and converting the voice information into text information to obtain the first text; The method further comprises: Recognizing the intention of the voice information; If the intention of the voice information is related to the first application, the first corresponding relationship is obtained, and the characters and suffix characters of the characters included in the first corresponding relationship are high-frequency words used in the first application.

9. The method according to claim 7 or 8, It is characterized in that The first application is a calendar application, and the first text is text related to creating a schedule; or, the first application is a gallery application, and the first text is text related to creating a video.

10. An electronic device, It is characterized in that include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A chip system, It is characterized in that The system comprises at least one processor and a communication interface, wherein the communication interface and the at least one processor are interconnected via a line, and the at least one processor is used to run a computer program or instruction to execute the method according to any one of claims 1 to 9.

13. A computer program product, It is characterized in that The invention comprises a computer program which, when being executed, causes a computer to execute the method according to any one of claims 1 to 9.