Handwritten text processing method and device, display system, equipment and medium

Abstract

The invention provides a handwritten text processing method and device, a display system, equipment and a medium, and relates to the technical field of image display. The handwritten text processing method provided by the invention comprises the following steps: generating a first handwritten text in response to a handwriting operation of a user; identifying the content of the first handwritten text to obtain a text identification result; generating a first translated text of the first handwritten text based on a language display requirement and the text recognition result; and displaying the first translated text. Through the handwritten text processing method, participants can translate handwritten content in a conference according to own language requirements, participants using other languages can accurately understand the meaning of the written content, and the conference participating experience of all participants is improved.

Description

Handwritten text processing method, device, display system, equipment and medium Technical Field

[0001] The present embodiment relates to the field of image display technology, and in particular to a handwritten text processing method, apparatus, display system, device, and medium. Background Art

[0002] In current conference and education scenarios, presenters may need to share handwritten content during a lecture. However, due to significant cultural differences among participants, attendees or listeners may have difficulty deciphering handwritten content, resulting in a poor conference experience.

[0003] Summary of the Invention

[0004] The present embodiment provides a handwritten text processing method, apparatus, display system, device, and medium to at least partially solve the problems existing in the related art.

[0005] A first aspect of an embodiment of the present invention provides a method for processing handwritten text, the method comprising:

[0006] generating a first handwritten text in response to a handwriting operation of a user;

[0007] Recognizing the content of the first handwritten text and obtaining a text recognition result;

[0008] generating a first translation text of the first handwritten text based on language display requirements and the text recognition result;

[0009] The first translated text is displayed.

[0010] Optionally, generating a first translation text of the first handwritten text based on the language display requirement and the text recognition result includes:

[0011] Determining a first text style of the first translated text, the first text style comprising at least one of the following: font, text color, and font size;

[0012] Based on the language display requirement and the first text style, a first translation text of the first handwritten text is generated.

[0013] Optionally, determining the first text style of the first translated text includes:

[0014] Display multiple candidate text styles;

[0015] In response to a text style selection operation by a user, a candidate text style selected by the user is determined as the first text style.

[0016] Optionally, determining the first text style of the first translated text includes:

[0017] determining a handwritten text style of the first handwritten text;

[0018] Based on the handwritten text style, a first text style of the first translated text is determined.

[0019] Optionally, determining a first text style of the first translated text based on the handwritten text style includes:

[0020] Matching the handwritten text style with multiple standard text styles, and determining the standard text style with the highest matching degree as the first text style; or

[0021] The handwritten text style is determined as the first text style.

[0022] Optionally, the method is applied to a display system, the display system comprising a first device and a second device, the first device and the second device being communicatively connected; the method further comprising:

[0023] The first device displays the text recognition result according to the requirements set by the second device;

[0024] Generating a first translation text of the first handwritten text based on the language display requirement and the text recognition result includes:

[0025] The first device changes the text recognition result into a first translation text that meets the language display requirements.

[0026] Optionally, the method is applied to a display system, the display system comprising a screen projection sending end and a screen projection receiving end, the screen projection sending end and the screen projection receiving end being communicatively connected; the displaying of the first translated text comprises:

[0027] The screen projection sending end sends the first screen projection image to the screen projection receiving end;

[0028] The screen projection receiving end uses the first screen projection image as a first base image and displays the first translated text on the first base image.

[0029] Optionally, the method is applied to a display system, the display system comprising a first device and a plurality of second devices, the first device and the plurality of second devices being communicatively connected; the method further comprising:

[0030] The plurality of second devices generate a plurality of first handwritten texts in response to a handwriting operation of a user;

[0031] The plurality of second devices recognize contents of the first handwritten texts generated by the respective devices, and obtain a plurality of text recognition results;

[0032] The multiple second devices generate first translated texts of the multiple first handwritten texts based on their respective language display requirements and text recognition results, and send the translated texts to the first device;

[0033] The first device displays the first translated texts of the multiple first handwritten texts in a paginated manner with multiple tabs.

[0034] Optionally, the first device displays the first translated texts of the multiple first handwritten texts in a plurality of tabbed pages, including:

[0035] In a case where the first device includes a large-screen terminal, in response to a user's selection operation on the large-screen terminal, determining a target tab from the plurality of tabs;

[0036] Based on the target tab, a first translation text corresponding to the target tab is displayed.

[0037] Optionally, the method further includes:

[0038] In a case where the first device further includes a mobile terminal in communication connection with the large-screen terminal, the large-screen terminal controls the mobile terminal to display the first translated text corresponding to the target tab.

[0039] Optionally, the method is applied to a display system, the display system including a first device and a second device, the first device being a handwriting device, and the first device and the second device being communicatively connected; generating the first handwritten text in response to a handwriting operation of a user includes:

[0040] The first device displays a second projection screen;

[0041] The first device generates, in response to a handwriting operation of a user, a first handwritten text associated with the second projection screen;

[0042] The method further comprises:

[0043] The first device sends the second projection image and the first handwritten text to the second device;

[0044] The displaying of the first translated text includes:

[0045] The second device displays the second projection screen and a first translation text of the first handwritten text generated by the second device.

[0046] Optionally, the method is applied to a display system, the display system including a first device and a second device, the first device and the second device being communicatively connected; generating the first handwritten text in response to a handwriting operation of a user includes:

[0047] The first device displays a second projection screen;

[0048] The first device generates, in response to a handwriting operation of a user, a first handwritten text associated with the second projection screen;

[0049] Generating a first translation text of the first handwritten text based on the language display requirement and the text recognition result includes:

[0050] The first device translates the text recognition result into a first translated text of the first handwritten text based on the language display requirement and the text recognition result;

[0051] The method further comprises:

[0052] The first device sends the first translated text to the second device;

[0053] The displaying of the first translated text includes:

[0054] The second device displays the first translated text;

[0055] The second device re-converts the first translated text in response to a conversion operation by the user, and displays the re-converted text.

[0056] Optionally, the method further includes:

[0057] The first device uses the second projection screen as a second background image and displays the first translated text on the second background image;

[0058] The first device sending the first translated text to the second device includes:

[0059] The first device sends the first translated text to the second device in response to the publishing instruction.

[0060] Optionally, generating the first handwritten text in response to the user's handwriting operation includes:

[0061] Display the third projection screen;

[0062] In response to a handwriting operation of the user, generating a first handwritten text associated with the third projection screen;

[0063] Generating a first translation text of the first handwritten text based on the language display requirement and the text recognition result includes:

[0064] Performing text recognition on the third projection screen to obtain a projection screen recognition result;

[0065] Through a natural language processing model, based on the language display requirements and the projection screen recognition results, the text recognition results are translated to obtain the first translated text.

[0066] Optionally, generating a first translation text of the first handwritten text based on the language display requirement and the text recognition result includes:

[0067] Based on the language display requirements, the text recognition results are translated to obtain and display multiple candidate translation texts that meet the language display requirements;

[0068] In response to a user's selection operation, the first translation text is determined from among the plurality of candidate translation texts.

[0069] Optionally, generating a first translation text of the first handwritten text based on language display requirements and the text recognition result includes:

[0070] Based on the language display requirement, translating the text recognition result into a converted intermediate text that meets the language display requirement;

[0071] Performing text style conversion on the converted intermediate text character by character to obtain the first translated text.

[0072] Optionally, the method further includes:

[0073] Acquiring target voice information synchronized with the first handwritten text;

[0074] Generating a first translation text of the first handwritten text based on the language display requirement and the text recognition result includes:

[0075] Recognizing the target voice information and obtaining a voice recognition result;

[0076] The text recognition result is translated by a natural language processing model based on the language display requirement and the speech recognition result to obtain the first translated text.

[0077] A second aspect of an embodiment of the present invention provides a method for processing handwritten text, the method comprising:

[0078] Get the first handwritten text;

[0079] Recognizing the content of the first handwritten text and obtaining a text recognition result;

[0080] generating a first translation text of the first handwritten text based on language display requirements and the text recognition result;

[0081] The first translated text is displayed.

[0082] A third aspect of an embodiment of the present invention provides a handwritten text processing device, the device comprising:

[0083] A text generation module, configured to generate a first handwritten text in response to a handwriting operation of a user;

[0084] A text recognition module, configured to recognize the content of the first handwritten text and obtain a text recognition result;

[0085] a text translation module, configured to generate a first translation text of the first handwritten text based on language display requirements and the text recognition result;

[0086] A text display module is used to display the first translated text.

[0087] A fourth aspect of an embodiment of the present invention provides a handwritten text processing device, the device comprising:

[0088] A text acquisition module, used for acquiring a first handwritten text;

[0089] a content recognition module, configured to recognize the content of the first handwritten text and obtain a text recognition result;

[0090] A text conversion module, configured to generate a first translation text of the first handwritten text based on language display requirements and the text recognition result;

[0091] A text display module is used to display the first translated text.

[0092] A fifth aspect of an embodiment of the present invention provides a display system, comprising: a first device and a second device, wherein the first device is communicatively connected to the second device; and when the first device and / or the second device are executed, the steps of the handwritten text processing method of the first aspect or the second aspect of the embodiment of the present invention are implemented.

[0093] A sixth aspect of an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the handwritten text processing method of the first aspect or the second aspect of the embodiment of the present invention are implemented.

[0094] A seventh aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the handwritten text processing method of the first aspect or the second aspect of the embodiment of the present invention are implemented.

[0095] This embodiment proposes a handwritten text processing method. After generating a first handwritten text in response to a user's handwriting operation, text recognition is performed on the first handwritten text to obtain a text recognition result. The text recognition result is then translated based on the set language display requirements to obtain a first translated text corresponding to the first handwritten text and display it. This allows participants to translate the handwritten content in the meeting according to their own language requirements, and allows participants who use other languages ​​to accurately understand the meaning of the written content, thereby improving the meeting experience of all participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the technical solution of this embodiment, the following briefly introduces the drawings required for use in the description of this embodiment. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0097] FIG1 is a flow chart of a handwritten text processing method according to the present embodiment;

[0098] FIG2 is a schematic diagram of a paginated display of handwritten content proposed in an embodiment of the present invention;

[0099] FIG3 is a schematic diagram of an interface of a mobile terminal proposed in an embodiment of the present invention;

[0100] FIG4 is a schematic diagram of an interface of a large-screen terminal proposed in an embodiment of the present invention;

[0101] FIG5 is a diagram of a confirmation interface of a large-screen terminal proposed in an embodiment of the present invention;

[0102] FIG6 is a confirmation diagram of an interactive interface of a mobile terminal proposed in an embodiment of the present invention;

[0103] 7 is a schematic diagram of a display interface of a mobile terminal proposed in an embodiment of the present invention;

[0104] FIG8 is a schematic diagram of a display interface of a mobile terminal provided in an embodiment of the present invention;

[0105] FIG9 is a schematic diagram of a text box provided by an embodiment of the present invention;

[0106] FIG10 is a schematic diagram of a viewing interface of a mobile terminal provided in an embodiment of the present invention;

[0107] FIG11 is a schematic diagram of a viewing interface editing method of a mobile terminal provided by an embodiment of the present invention.

[0108] FIG12 is a flow chart of a handwritten text processing method according to the present embodiment;

[0109] FIG13 is a block diagram of a handwritten text processing apparatus according to an embodiment of the present invention;

[0110] FIG14 is a block diagram of a handwritten text processing apparatus according to an embodiment of the present invention;

[0111] FIG15 is a schematic diagram of an electronic device according to an embodiment of the present invention. Specific embodiments

[0112] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0113] This embodiment can be applied to conference or teaching scenarios, such as large or medium-sized conferences involving multiple participants using different languages ​​or expressions. When participants need to share content during the meeting and need to handwrite content on the terminal device to assist in expressing themselves, if the handwritten content is displayed directly as is, other participants may not be able to recognize it or have difficulty recognizing it.

[0114] Based on this, this embodiment proposes a handwritten text processing method, which generates a first handwritten text according to the user's handwriting operation and performs text recognition to obtain a text recognition result; the text recognition result is translated based on the set language display requirements to obtain a first translated text corresponding to the first handwritten text and display it, so that participants can translate and process the handwritten content in the meeting according to their own needs, so that participants who use other languages ​​can accurately understand the meaning of the written content, thereby improving the meeting experience of all participants.

[0115] 1 , a flow chart of a handwritten text processing method according to this embodiment is shown. The handwritten text processing method provided by this embodiment may include the following steps:

[0116] S101 : Generate a first handwritten text in response to a handwriting operation of a user.

[0117] In this embodiment, during a meeting or teaching process, a user can publish handwritten content, and in response to the user's handwriting operation, a first handwritten text is determined. The first handwritten text can be a plurality of track points collected based on the user's handwriting operation, or a collected image containing the plurality of track points.

[0118] The display device includes a touch-sensitive component. When a user performs handwriting operations on the display device, the touch-sensitive component is configured to obtain a sequence of touch track points of the user's handwriting input. A system-on-chip (SOC) is in communication with the touch-sensitive component. The SOC can obtain the sequence of touch track points through the touch-sensitive component and generate a first handwritten text based on the sequence of touch track points. Furthermore, in a preferred embodiment, the SOC can smooth / beautify the handwriting when generating the first handwritten text, or adjust the line width of the handwriting based on speed or pressure, thereby ultimately generating the first handwritten text.

[0119] S102: Recognize the content of the first handwritten text and obtain a text recognition result.

[0120] In this embodiment, handwritten text recognition can be performed on the obtained first handwritten text, for example, by using a handwritten text recognition algorithm (such as a GCN, RNN, or OCR algorithm) to recognize the content of the first handwritten text to obtain a text recognition result. The text recognition result in this embodiment is a text in a first language generated and recognized in response to the user's handwriting operation. The first language can be any language, such as Chinese, English, etc., without limitation.

[0121] S103: Generate a first translation text of the first handwritten text based on language display requirements and the text recognition result.

[0122] S104: Display the first translated text.

[0123] In this embodiment, after obtaining the text recognition result of the first handwritten text, the text recognition result can be translated based on the set language display requirements, and the text recognition result can be translated into a first translated text that meets the language display requirements, and the first translated text can be displayed. The first translated text is a text in the second language. Preferably, the first translated text can be displayed in a standard font of the second language.

[0124] It should be noted that the first translated text in this embodiment can be an image, i.e., an image containing text in the second language. Therefore, when displaying the first translated text, a layer can be generated, an image of the first translated text can be added to the layer, and the layer can be controlled for display. For example, a method for generating a new layer in Android includes: creating a new view in Android and drawing a note on the view's canvas; a method for closing a layer in Android includes: if the layer is on an Android view, directly closing the view.

[0125] The user can set the language display requirement in advance or in real time. That is, the language display requirement in this embodiment can be a default language display requirement pre-set based on the user's previous language selection operations, or it can be a real-time language display requirement determined based on the user's current language selection operation. This embodiment does not limit this. The language display requirement is intended to translate the text recognition result of the first handwritten text into a first translated text.

[0126] It is understandable that the first language is the original language generated by handwriting, and the second language is the presentation language in the meeting. The first language and the second language may be the same or different.

[0127] Among them, when the first language and the second language are the same, it indicates that there is no cultural barrier between users in the meeting. At this time, after obtaining the text recognition result, it can be determined whether the first language corresponding to the text recognition result is the same as the second language in the language display requirement. If it is determined that they are not the same, translation processing is performed. If it is determined that they are the same, the text recognition result is directly displayed, or the text recognition result is converted into a standard font and then displayed, or the text recognition result is still translated into the first translation text of the second language and displayed.

[0128] When a user inputs handwritten content during a meeting, this embodiment can generate a first handwritten text in response to the user's handwriting operation, recognize the handwritten content, and translate and display the handwritten content according to the set language display requirements. This allows participants who speak other languages ​​to accurately understand the meaning of the written content through the receiving end's recognition, achieving real-time and accurate translation and display of the handwritten content, thereby improving the user's meeting experience. In addition, if there are remote participants, they can also view the translated content remotely on their mobile devices.

[0129] In combination with the above embodiments, this embodiment provides a handwritten text processing method. In this method, step S103 may include sub-steps S201 and S202:

[0130] S201 : Determine a first text style of the first translated text, where the first text style includes at least one of the following: font, text color, and font size.

[0131] In this embodiment, a first text style of the first translated text can be determined. The first text style is the display style of the first translated text, and is intended to display the first translated text in the first text style. The first text style includes at least one of the following: font, text color, font size, thickness, etc.

[0132] S202: Generate a first translation text of the first handwritten text based on the language display requirement and the first text style.

[0133] In this embodiment, the text recognition result of the first handwritten text can be translated based on the language display requirements to obtain the translated text, and then the translated text can be converted based on the first text style to generate a first translated text of the first handwritten text that conforms to the first text style. In this embodiment, not only can the handwritten content in the meeting be translated instantly to obtain the first translated text, but the first translated text can also be displayed according to the set text style, thereby greatly facilitating the digital organization and review of the teaching content, and making it easier for other participants or listeners to read.

[0134] Furthermore, the text recognition results can be converted to text format based on the set text format display requirements, and the text recognition results can be converted to text that meets the text display format requirements and displayed. Similar to the language display requirements, the text format display requirements can be default text format display requirements pre-set based on the user's previous text format selection operations, or real-time text format display requirements determined based on the user's current text format selection operation. This embodiment does not impose any restrictions on this.

[0135] In this embodiment, based on the fact that the handwriting of different people varies greatly in conference and education scenarios, which may cause recognition difficulties for participants, this embodiment proposes to recognize handwritten text and convert it into standard computer fonts for display. In this way, the content inserted by the lecturer in handwriting during the lecture (i.e., the first handwritten text) can be greatly facilitated by converting the text recognition results into text format such as standard text, which can greatly facilitate the digital organization and review of the lecture content and make it easier for other participants or listeners to read. In addition, in the case where there are large language or cultural differences among the participants, the handwritten content inserted in real time in this embodiment can also be translated into other languages ​​or expressions with high quality to improve the experience of all participants or listeners.

[0136] In combination with the above embodiments, this embodiment provides a handwritten text processing method. In this method, the above step S201 may specifically include sub-steps S301 and S302:

[0137] S301, displaying multiple candidate text styles;

[0138] S302 : In response to a text style selection operation by a user, determining a candidate text style selected by the user as the first text style.

[0139] This embodiment displays multiple candidate text styles, and the user can make a selection based on the multiple candidate text styles. In response to the user's text style selection operation, one candidate text style selected by the user from the multiple candidate text styles is determined as the first text style of the first translation text, and the first translation text is displayed in the first text style.

[0140] In this embodiment, the user can select a candidate text style, and the first translated text is displayed based on the selected candidate text style.

[0141] In combination with the above embodiments, this embodiment further provides a handwritten text processing method. In this method, the above step S201 may specifically include sub-steps S401 and S402:

[0142] S401: Determine the handwritten text style of the first handwritten text.

[0143] In this embodiment, the text style of the first handwritten text can be identified based on the generated first handwritten text. For example, the style of the first handwritten text can be identified through a pre-trained text style recognition model to obtain the handwritten text style corresponding to the first handwritten text, wherein the handwritten text style is a text style vector and exists in the form of a vector.

[0144] For example, the text style recognition model of this embodiment may be a style encoding model, such as a VGG network, etc., which is not limited in this embodiment.

[0145] S402: Determine a first text style of the first translated text based on the handwritten text style.

[0146] In this embodiment, the first text style of the first translated text may be determined based on the handwritten text style, so that the first translated text finally displayed may be adapted to the handwriting of the user's first handwritten text.

[0147] Specifically, in a preferred embodiment, the above step S402 may include sub-step S501 or sub-step S502:

[0148] S501 : Match the handwritten text style with a plurality of standard text styles, and determine a standard text style with the highest matching degree as the first text style.

[0149] In this step, the determined handwritten text style can be matched with multiple standard text styles to obtain a degree of match between each standard text style and the handwritten text style. The standard text style with the highest degree of match is then determined as the first text style of the first translated text. In other words, the first translated text is displayed using the standard text style that is closest to the handwritten style. For example, the handwritten text style of the first handwritten text has the highest degree of match with Kaiti, so Kaiti is used as the first text style for displaying the first translated text.

[0150] For example, when comparing the handwritten text style with the standard text style, the image of the standard text can be input into the style encoding model to obtain the standard text style output by the style encoding model (i.e., the style vector of the standard text), and then the matching degree between the handwritten text style vector and the standard text style vector is calculated.

[0151] S502: Determine the handwritten text style as the first text style.

[0152] In this step, the determined handwritten text style may be directly determined as the first text style and the first translated text may be directly displayed.

[0153] In combination with the above embodiments, this embodiment provides a handwritten text processing method. In this method, the method is applied to a display system, and the display system includes a first device and a second device.

[0154] In this embodiment, the first device and the second device are participating terminals in the same conference scene or teaching scene. Among them, the first device is the receiving end of the handwritten content in the conference scene or teaching scene, and the second device is the sending end of the handwritten content in the conference scene or teaching scene, and a communication connection is established between the first device and the second device. In this embodiment, the second device can be any participating terminal, or it can be a participating terminal with handwritten content sharing authority, and there is no restriction on this. As for the first device, the first device can be any participating terminal other than the second device, or it can be a participating terminal with handwritten content sharing authority other than the second device, and these can be arbitrarily set according to needs, and this embodiment does not limit this.

[0155] In addition to the above steps, the method may further include step S601:

[0156] S601: The first device displays the text recognition result according to the requirements set by the second device.

[0157] In this embodiment, the second device generates a first handwritten text in response to a user's handwriting operation, recognizes the content of the first handwritten text, obtains a text recognition result, and then sends the text recognition result to the first device. After the first device receives the text recognition result, the first device may first display the received text recognition result according to the requirements set by the second device. The text recognition result received by the first device may carry the display requirements set by the second device. For example, based on the user's setting operation, the second device sets corresponding display requirements for the text recognition result generated by the recognition. The display requirements include at least: the language display requirements of the second device and / or the text format display requirements of the second device. The second device sends the generated text recognition result and the set display requirements to the first device, and the first device displays the text recognition result based on the text recognition result and the display requirements of the second device. Alternatively, after the second device generates the text recognition result locally, it directly converts the text recognition result based on the set display requirements and then sends the converted text recognition result to the first device, so that the first device directly displays the converted text recognition result.

[0158] Furthermore, in this embodiment, the step S103 may specifically include sub-step S602:

[0159] S602: The first device changes the text recognition result into a first translation text that meets the language display requirement.

[0160] In this embodiment, after displaying the text recognition result according to the requirements set by the second device, the first device can change the text recognition result to a first translated text that meets the language display requirements based on the language display requirements set by the user on the first device and display it.

[0161] In another embodiment, in response to the language display requirements and text format display requirements set by the user on the first device, the displayed text recognition results can be changed to: a first translated text that meets the language display requirements, and a text that meets the text display format requirements.

[0162] In this embodiment, the receiving end of the handwritten content (i.e., the first device) first displays the handwritten content (i.e., the first handwritten text) according to the requirements set by the handwriting end of the handwritten content (i.e., the second device), and then converts and displays the handwritten content based on the language display requirements of the receiving end.

[0163] In combination with the above embodiments, this embodiment provides a handwritten text processing method, in which the method is applied to a display system, the display system includes a screen projection sending end and a screen projection receiving end, and the screen projection sending end and the screen projection receiving end are communicatively connected; the step S104 may specifically include sub-steps S701 and S702:

[0164] S701, the screen projection sending end sends the first screen projection picture to the screen projection receiving end.

[0165] In the meeting of this embodiment, screen projection can be performed to realize sharing of the projection screen. The screen projection sending end can send the first projection screen provided by itself to the screen projection receiving end.

[0166] S702: The screen projection receiving end uses the first projection screen as a first background image and displays the first translated text on the first background image.

[0167] In this embodiment, the screen projection receiving end can receive the first projection image provided by the screen projection sending end. In addition to providing the first projection image, the screen projection receiving end also sends the first translation text in real time. Thus, after receiving the first translation text, the screen projection receiving end in this embodiment uses the first projection image provided by the screen projection sending end as the first background image and displays the first translation text on the first background image.

[0168] Furthermore, when the screen projection sending end is set with language display requirements and text format display requirements, after the screen projection receiving end receives the first projection screen, the first translated text and the text that meets the text format display requirements sent by the screen projection sending end, the first projection screen is used as the first base map, the first translated text is displayed on the first base map, and the text that meets the text format display requirements is displayed on the first base map.

[0169] In this embodiment, the projection screen (such as a shared file) provided by the projection sending end is used as the background image at the projection receiving end to display the first translated text, so that participants can have a better conference experience.

[0170] In conjunction with the above embodiment, in another embodiment, this embodiment provides a handwritten text processing method, which is applied to a display system. In this method, the display system includes a first device and multiple second devices, and the first device and the multiple second devices are communicatively connected. In addition to the above steps, steps S801 to S804 may also be included:

[0171] S801: The multiple second devices generate multiple first handwritten texts in response to a user's handwriting operation.

[0172] S802: The multiple second devices recognize the content of the first handwritten text generated by each of them, and obtain multiple text recognition results.

[0173] In this embodiment, during the conference, multiple second devices may simultaneously respond to the user's handwriting operation, generate multiple first handwritten texts, and each second device performs text recognition on the generated first handwritten texts to obtain multiple text recognition results.

[0174] S803: The multiple second devices generate first translated texts of the multiple first handwritten texts based on their respective language display requirements and text recognition results, and send the translated texts to the first device.

[0175] In this embodiment, multiple second devices can translate their respective text recognition results based on their respective set language display requirements and their respective generated text recognition results, thereby generating multiple first translated texts, and send the multiple first translated texts to the first device.

[0176] S804: The first device displays the first translated texts of the multiple first handwritten texts in a plurality of tabbed pages.

[0177] In this embodiment, after receiving multiple first translations, the first device can display the multiple first translations as tabbed pages. In other words, in this embodiment, multiple first translations that meet language display requirements are displayed as tabbed pages. Only one first translation sent by the second device is displayed on each tab page.

[0178] Furthermore, if the second device is configured with language display requirements and text format display requirements, after receiving multiple first translated texts and multiple texts that meet the text format display requirements from multiple second devices, the first device displays the multiple first translated texts and multiple texts that meet the text format display requirements in the form of multiple groups on multiple tabs during pagination display. Only one first translated text and text that meet the text format display requirements sent by the second device is displayed on each tab page.

[0179] In this embodiment, the first device can display multiple first translations published by multiple second devices in layers and tabs, as shown in Figure 2, which is a schematic diagram of a tabbed display of handwritten content proposed in an embodiment of the present invention. In Figure 2, three second devices have published real-time handwritten content. When the first device displays the first translation, three interactive tabs corresponding to different layers are displayed on the left side of the first device screen, respectively, to display different layers.

[0180] In combination with the above embodiments, this embodiment provides a handwritten text processing method. In this method, step S804 may specifically include sub-steps S901 and S902.

[0181] S901: When the first device includes a large-screen terminal, in response to a user's selection operation on the large-screen terminal, determine a target tab from the multiple tabs.

[0182] In this embodiment, the first device may include a large-screen end and / or a mobile terminal, wherein the large-screen end is a large display device, such as a large-screen end may refer to a display device larger than 30 inches, for example: 31.5 inches, 43 inches, 55 inches, 65 inches, 75 inches, 86 inches, etc. The specific type of the large-screen end may be an all-in-one conference machine, a television, a computer, etc.; the mobile terminal is a mobile handwriting device that supports handwriting, such as a tablet computer, a computer, a notebook, a mobile phone, etc., and this embodiment does not limit this.

[0183] In the case where the first device includes a large-screen terminal, the user can perform a selection operation on the large-screen terminal to determine the tab that needs to be displayed currently. In response to the user's selection operation on the large-screen terminal, the large-screen terminal can determine a target tab from multiple tabs.

[0184] S902: Based on the target tab, display a first translation text corresponding to the target tab.

[0185] In this embodiment, after determining the target tab, the large-screen terminal can display the first translated text corresponding to the target tab. If the first device is set with language display requirements and text format display requirements, the large-screen terminal can display the first translated text corresponding to the target tab and the text that meets the text format display requirements.

[0186] Furthermore, in combination with the above embodiments, this embodiment provides a handwritten text processing method. In this method, in addition to the above steps, the method may further include step S1003:

[0187] S1003: In a case where the first device further includes a mobile terminal in communication with the large-screen terminal, the large-screen terminal controls the mobile terminal to display a first translation text corresponding to the target tab.

[0188] In this embodiment, if the first device also includes a mobile terminal in communication with the large-screen terminal, that is, the first device includes the large-screen terminal and the mobile terminal, after the large-screen terminal determines the target tab, it can control the mobile terminal in communication with it to display the first translated text corresponding to the target tab. Furthermore, if the first device is configured with language display requirements and text format display requirements, the first device can control the mobile terminal in communication with it to display the first translated text corresponding to the target tab and text that meets the text format display requirements.

[0189] In this embodiment, when the receiving end of the handwritten content includes a large-screen end and a mobile terminal, the large-screen end determines the target tab among multiple tabs to display the handwritten content, and controls other mobile terminals to display the target tab.

[0190] In one embodiment, the first and second devices participating in the conference require at least one mobile handwriting device and one large display device. The mobile handwriting device (mobile terminal) includes a tablet computer and a stylus (e.g., an electronic pen), and the large display device (large-screen terminal) includes an all-in-one conference machine, both of which support Bluetooth. In this embodiment, the first device can be a large-screen terminal, and the second device can be a mobile terminal.

[0191] Mobile devices and large-screen devices can establish a short-range wireless connection via Bluetooth. To connect, ensure both devices support Bluetooth. Turn on Bluetooth on both devices. The mobile device searches for available Bluetooth devices, selects the large-screen device to connect to, and waits for the connection to be established. If a pairing code is required, enter a fixed-digit code, such as "0000" or "1234," and wait for pairing to complete.

[0192] To enable interactive display of handwritten content between a mobile terminal and a large-screen device, the same application software APP must be installed on both the mobile terminal and the large-screen device. The mobile terminal APP must support editing functions such as handwriting input, content editing, and layout layout, while the large-screen APP must at least support display functions. The APP of this embodiment supports multiple language modes (such as English, French, Russian, Arabic, etc.), and all corresponding interface text and control text are adapted.

[0193] Furthermore, in one embodiment, as shown in Figure 3, Figure 3 is a schematic diagram of the interface of a mobile terminal proposed in an embodiment of the present invention. In Figure 3, after the mobile terminal opens the APP, the menu bar above the mobile terminal interface provides main functions: such as the "large screen end" button, which is used to confirm the addition of the large screen end to be interacted; the "real-time handwriting recognition" button, which is used to turn on the handwriting recognition function; the "watch" button, which is used to view the handwritten content posted by other terminals. The lower right side is a preview area, such as a rectangular area with a dotted border, which previews the current public screen display content. The content presented in the preview area can be a source file, that is, a shared file, such as a PPT file, or it can be a screen projection mapping, that is, a file shared by other terminals, depending on whether the source file sharer has enabled collaboration permissions.

[0194] In one embodiment, as shown in Figure 4, which is a schematic diagram of a large-screen terminal interface proposed in an embodiment of the present invention, after the large-screen terminal opens the APP, the large-screen terminal interface can select a desktop for interactive display at the top, and all mobile terminals that have been connected via Bluetooth are displayed at the bottom.

[0195] Furthermore, in one embodiment, after the mobile terminal and the large-screen end have identified the device through Bluetooth, the mobile terminal and the large-screen end need to further confirm the permissions after opening the APP. Specifically, the large-screen end first determines the desktop that needs to be interacted with, and a dialog box pops up to display the interaction permission verification code, as shown in Figure 5, which is a confirmation diagram of the interaction interface of the large-screen end proposed in an embodiment of the present invention. In Figure 5, the large-screen end can wait for the mobile terminal to confirm to join the interaction, and in response to the user clicking the "Start" button, it starts to enter the interactive state, the APP on the large-screen end is minimized and exited, and runs in the background, and the interface of the large-screen end returns to the desktop being interacted with.

[0196] As for the mobile terminal, after the mobile terminal opens the app, in response to the user clicking the "large screen terminal" button in the menu bar, a dialog box pops up in the interface for entering the interactive permission verification code. As shown in Figure 6, Figure 6 is a confirmation diagram of the interactive interface of a mobile terminal proposed in an embodiment of the present invention. In Figure 6, after the mobile terminal determines that the verification is successful and the large screen terminal enters the interactive state, the interactive desktop selected by the large screen terminal will be directly projected to the preview area in the lower right corner of the mobile terminal to realize the desktop preview.

[0197] In combination with the above embodiments, this embodiment provides a method for processing handwritten text. In addition to the above steps, the method may also include the following steps:

[0198] Step 1: Determine the edit text in response to the user's editing operation on the first projection screen.

[0199] In this embodiment, the first device is a synchronous writing device authorized by the second device. The first device has permission to edit the first projection screen provided by the second device. That is, the second device and the first device share the same handwriting interface (the first projection screen). The first device can determine the edited text in response to the user's editing operation on the first projection screen.

[0200] Step 2: Based on the language display requirement, the edit text is translated to generate and display the edit text that meets the language display requirement.

[0201] In this embodiment, after the first device recognizes the edit text, it can translate the edit text based on the language display requirements of the first device to generate and display the edit text that meets the language display requirements.

[0202] Step 3: Send the edited text to the second device.

[0203] In this embodiment, after determining that the edited text is obtained, the edited text can be sent to the second device; or after determining that the edited text meets the language display requirements, the edited text that meets the language display requirements can be sent to the second device for display by the second device. This embodiment does not limit this.

[0204] Furthermore, if the first device is configured with language display requirements and text format display requirements, the first device may translate the edited text to generate and display edited text that meets the language display requirements, and then perform text format conversion on the edited text to generate and display edited text that meets the text format display requirements. Furthermore, the edited text that meets the language display requirements and the edited text that meets the text format display requirements may be sent to the second device for display by the second device.

[0205] In this embodiment, in a collaborative editing scenario, the first device and the second device share a handwriting interface, and multiple terminals can perform writing and editing, and multiple terminals can display simultaneously.

[0206] In combination with any of the above embodiments, this embodiment provides a method for processing handwritten text, wherein the method is applied to a display system, the display system including a first device and a second device, the first device being a handwriting device, and the first device and the second device being communicatively connected. Step S101 above may specifically include the following sub-steps S1101 and S1102:

[0207] S1101, the first device displays the second projection screen.

[0208] S1102: The first device generates a first handwritten text associated with the second projection screen in response to the user's handwriting operation. In the display system, the first device can serve as a receiver or a transmitter of handwritten content. That is, the user can perform handwriting operations on the second device and then send the text recognition results to the first device for display of the translated content; or the user can perform handwriting operations on the first device and then send the text recognition results to the second device for display.

[0209] Specifically, in this embodiment, the first device is a handwriting device, and the first device can display a second projection screen, where the second projection screen is content shared by the first device. The first device can generate first handwritten text associated with the second projection screen in response to a user's handwriting operation on the second projection screen.

[0210] In addition to the above steps, this method may further include step S1103; and the above step S104 may specifically include sub-step S1104:

[0211] S1103, the first device sends the second projection image and the first handwritten text to the second device.

[0212] S1104: The second device displays the second projection screen and a first translation text of the first handwritten text generated by the second device.

[0213] In this embodiment, after the first device generates the first handwritten text associated with the second projection screen, it can send the second projection screen and the first handwritten text associated with the second projection screen to the second device so that the second device can view the handwritten content.

[0214] After the second device receives the second projection screen and the first handwritten text, it can first perform text recognition on the first handwritten text to obtain a text recognition result, and then translate the text recognition result based on the language display requirements set by the second device to obtain a first translated text, so that the second device can display the second projection screen and the first translated text generated by the second device.

[0215] For example, a user may perform handwriting operations on a mobile handwriting device (second device), and then send the handwritten content to a large display device (first device) for display, and simultaneously display the translated content on the large display device; in another embodiment, handwriting may be performed on the large screen (first device), and then the handwritten content and translated content may be sent to a mobile handwriting device (second device) for display.

[0216] Specifically, before performing a handwriting operation, the user can perform a click operation on the screen of the first device. In response to the user's click operation, the first device can display a handwriting collection area on the screen of the first device. The user can perform a handwriting operation in the handwriting collection area using a handwriting capacitive pen or finger. The first screen generates second text content in response to the handwriting operation in the handwriting collection area.

[0217] In one embodiment, as shown in Figure 7, which is a schematic diagram of a display interface of a mobile terminal proposed in an embodiment of the present invention, after a user selects the "Real-time Handwriting Recognition" function on the mobile terminal, the mobile terminal responds to the user's click operation by displaying a handwriting collection area on the left side of the screen (such as the rectangular area with a dashed border on the left side of Figure 7) and a preview area on the right side. The desktop content previously selected for display on the large screen can be completely projected into the preview area as a base map.

[0218] To recognize handwritten content, users can use their fingers or a stylus to input in the handwriting collection area. The mobile terminal's handwriting recognition algorithm extracts and intelligently analyzes the input handwriting points within the handwriting collection area. The handwriting collection area can have the following properties: 1. Input function: What you see is what you write; 2. Deletion function: Historical strokes can be deleted or erased; 3. Lasso function: Used to adjust the front and back position of handwritten text.

[0219] In combination with any of the above embodiments, a handwritten text processing method of this embodiment is applied to a display system, wherein the display system includes a first device and a second device, wherein the first device and the second device are in communication connection. In this method, the above step S101 may specifically include steps S1201 and S1202, the above step S103 may specifically include step S1203, the above step S104 may specifically include steps S1205 and S1206, and, in addition to the above steps, step S1204 may also be included.

[0220] S1201, the first device displays the second projection screen.

[0221] S1202: The first device generates a first handwritten text associated with the second projection screen in response to a handwriting operation of the user.

[0222] In this embodiment, step S1201 and step S1202 are the same as or similar to the above-mentioned step S1101 and step S1102, and are not described in detail in this embodiment.

[0223] S1203: The first device translates the text recognition result into a first translated text of the first handwritten text based on the language display requirement and the text recognition result.

[0224] In this embodiment, after the first device generates the first handwritten text, it can perform text recognition on the first handwritten text to obtain a text recognition result, and then translate the text recognition result based on the language display requirements set by the first device to obtain a first translated text of the first handwritten text.

[0225] S1204: The first device sends the first translated text to the second device.

[0226] In this embodiment, after obtaining the first translated text, the first device may send the first translated text to the second device for display by the second device.

[0227] S1205: The second device displays the first translated text.

[0228] In this embodiment, the second device may directly display the received first translated text.

[0229] S1206: The second device re-converts the first translated text in response to the user's conversion operation, and displays the re-converted text.

[0230] In this embodiment, after a user views a first translated text on a second device, if the language display requirements of the first translated text do not meet the user's needs, the user can re-translate the first translated text on the first device. Specifically, the second device can perform a conversion operation on the first translated text, and the second device, in response to the user's conversion operation, re-converts the first translated text and displays the re-converted text.

[0231] For example, the first translated text generated by the first device is an English text. After viewing the English text on the second device, the user finds that he cannot understand it. At this time, the English text can be translated on the second device, such as translating the English text into French text, and then displaying the translated text (French text).

[0232] In this embodiment, after the first device translates the handwritten text based on its own language display requirements and sends it to the second device, if the user is not satisfied with the first translated text displayed by the second device, the second device can retranslate the first translated text and display the re-converted text to improve the user's conference experience of the second device.

[0233] In combination with any of the above embodiments, a handwritten text processing method of this embodiment may further include the following step S1301 in addition to the above steps, and the step S1203 may include sub-step S1302:

[0234] S1301: The first device uses the second projection screen as a second background image and displays the first translated text on the second background image.

[0235] In this embodiment, there is a preview area on the screen of the first device. After the first device recognizes the first handwritten text and translates it to obtain the first translated text, the first translated text can be displayed on the second base map with the second voting screen as the second base map in the preview area, thereby realizing a preview of the current public screen display content.

[0236] S1302: The first device sends the first translated text to the second device in response to the publishing instruction.

[0237] In this embodiment, the user can determine whether to send the first translated text based on the content of the preview area. If the user wants to make further changes to the first translated text, the user can perform editing operations on the first translated text in the preview area. For example, the display content of the preview area is selectable and editable (direct addition and deletion, modification of font, color, text direction, etc.), or the user can freely select the language in the text box of the preview area, which can be set to English by default. In response to the editing operation, the first device can perform further required conversion on the first translated text, such as further changing the text format or language translation, thereby obtaining a new first translated text. After the user is satisfied, the user can perform a publishing operation on the first device, and the first device sends the first translated text to the second device in response to the publishing instruction.

[0238] In this embodiment, a preview area is provided on the first device when the user performs a handwriting operation. The generated first handwritten text and the first translated text can be displayed in the preview area. The user can make a publishing confirmation based on the content displayed in the preview area, thereby sending the first translated text, thereby improving the accuracy and display effect of the first translated text.

[0239] In one embodiment, the first device is a mobile terminal, which uses a handwriting recognition algorithm to collect input content in a handwriting collection area in real time, analyzes it in real time, and obtains a standard font code corresponding to the handwriting. Specifically, the mobile terminal may have an app with a built-in handwriting recognition algorithm. The app can convert the handwriting into a standard computer font based on the standard font code obtained by the handwriting recognition algorithm, and preview the recognition results in real time in a preview area to the right of the handwriting collection area.

[0240] As shown in Figure 8, Figure 8 is a schematic diagram of a display interface of a mobile terminal provided by an embodiment of the present invention. In Figure 8, the dotted area on the right side of the screen is the preview area, and different preview areas display content in different languages ​​converted / translated by the recognized font. The preview area can have the following attributes:

[0241] 1. Number of text boxes: The default is 2, and you can also add, expand, or remove text boxes based on actual needs, as shown in Figure 9, which is a schematic diagram of a text box provided by an embodiment of the present invention. In Figure 9, the basic recognition result can be defaulted to a Chinese text box. The plus button on the right means that you can add a new text box below, and you can choose other fonts or languages. To the right of the newly added text box, you can continue to add languages ​​using the plus sign or delete text boxes using the minus sign.

[0242] 2. Text box position and shape: The text box shape can be expanded, the text box position should be draggable, and pictures can also be displayed directly.

[0243] 3. Content: The content in the preview area can be selected and edited (directly add or delete, modify font, color, text direction, etc.).

[0244] 4. Language selection: Users can freely select the language in the recognition result text box in the preview area, and English can be the default.

[0245] 5. Style: You can choose fonts from the standard font library or use fonts generated by the intelligent handwriting style. There is no restriction on this.

[0246] Once the mobile user has completed handwritten text recognition and translation and adjusted the desired position and style in the preview area, they can publish it to the large screen. Furthermore, after the mobile user publishes the recognized and translated content, other participants can watch it on their mobile devices in addition to the large screen. Therefore, the mobile device must be in viewing mode.

[0247] When the mobile terminal user clicks the "Watch" button in the menu bar of the mobile terminal interface, the mobile terminal enters the viewing mode. At this time, the content displayed in the preview area below the mobile terminal interface is synchronized with the content displayed on the large screen, as shown in Figure 10. Figure 10 is a schematic diagram of a viewing interface of a mobile terminal provided by an embodiment of the present invention. In Figure 10, when the mobile terminal is in the current viewing mode, the mobile terminal user can view the translated content published by other mobile terminals. At the same time, the translated content published by other mobile terminals can be locally re-translated as needed. As shown in Figure 11, Figure 11 is a schematic diagram of editing the viewing interface of a mobile terminal provided by an embodiment of the present invention. In Figure 11, the user can select the content that needs to be re-translated in the viewing interface of the mobile terminal, and a text box similar to that in Figure 9 will appear. The plus button on the right side of the text box can select a new translation language, but at this time the originally selected content is read-only and cannot be modified.

[0248] In one embodiment, the handwritten text recognition algorithm of this embodiment is implemented in the application APP on the mobile terminal side. Specifically, the content of the handwriting collection area is intercepted at a fixed frequency, and the sampling result is in the form of a picture. Two deep learning models of text detection and text recognition are used to complete the recognition task.

[0249] Among them, the text detection model of this embodiment (taking the CTPN algorithm as an example), for the sample picture of the input handwriting collection area, first extracts image features through the backbone (such as the VGG network) to obtain an image feature map. Then, a sliding window feature convolution is performed on the image feature map through conv5, that is, each point is combined with the surrounding area features to obtain a feature vector. This feature vector feature map is then input into the bidirectional LSTM network as a data stream to learn the sequence features of each row. The output of the bidirectional LSTM is then restored to a shape based on the original image by Reshape. At this time, the feature contains both spatial features and sequence features learned by LSTM. Then, the "FC" convolution layer is used to further extract features, and finally, an RPN network similar to Faster R-CNN is used to obtain text proposals and obtain a text detection box.

[0250] The network structure of the text recognition model of this embodiment (taking the CNN+RNN+CTC algorithm as an example) includes three parts, from bottom to top: CNN (convolutional layer): using deep CNN to extract features from the input image and obtain a feature map; RNN (recurrent layer): using a bidirectional RNN (BLSTM) to predict the feature sequence, learn each feature vector in the sequence, and output the predicted label (true value) distribution; CTC loss (transcription layer): using CTC loss to convert the series of label distributions obtained from the recurrent layer into the final text sequence.

[0251] In one embodiment, the translation of handwritten text is achieved through a deep learning language translation algorithm, which is implemented in an application APP on the mobile terminal side. Specifically, based on the results of handwritten text recognition, the recognized content is used as the input text sequence, and the translation content corresponding to different languages ​​is output. Among them, a general natural language processing translation model (taking transformer as an example) includes an encoder and a decoder. In the encoder-decoder architecture, the encoder converts the input text sequence into a continuous expression, and then the decoder generates an output sequence text based on the expression, thereby achieving the translation task of variable-length input and output.

[0252] In combination with any of the above embodiments, this embodiment provides a handwritten text processing method. In this method, step S101 may specifically include sub-steps S1401 and S1402, and step S103 may include sub-steps S1403 and S1404:

[0253] S1401, display the third projection screen.

[0254] S1402, in response to a user's handwriting operation, generate a first handwritten text associated with the third screen mirroring image.

[0255] In this embodiment, the third screen mirroring image can be displayed, and the user can perform a handwriting operation based on the third screen mirroring image. In response to the user's handwriting operation, a first handwritten text associated with the third screen mirroring image is generated.

[0256] S1403, perform optical character recognition on the third screen mirroring image to obtain an optical character recognition result of the screen mirroring image.

[0257] S1404, through a natural language processing model, based on the language display requirement and the optical character recognition result of the screen mirroring image, perform translation processing on the optical character recognition result to obtain the first translated text.

[0258] In this embodiment, for the problem that the content of the recognized second handwritten text is scarce and translation inaccuracies are likely to occur. For example, when the user writes the character "王" (Wang), it may have two semantic meanings: (1) king and (2) surname Wang. Based on this, this embodiment proposes a method for judging the context semantics according to the content of the background material.

[0259] Specifically, in this embodiment, when the optical character recognition result corresponding to the first handwritten text includes a number of characters below a preset threshold, optical character recognition is performed on the third screen mirroring image to obtain an optical character recognition result of the screen mirroring image. Then, the optical character recognition result of the third screen mirroring image and the optical character recognition result are input into a pre-trained natural language processing model (for example: a natural language processing model based on prompt learning, such as GPT2, GPT3.5, GPT4, LLaMA, or ChatGLM model, etc.). Through the pre-trained natural language processing model, based on the language display requirement, translation processing is performed on the optical character recognition result, and the optical character recognition result of the third screen mirroring image is used as a prompt word for translation hint to obtain the first translated text that meets the language display requirement. <s

[0260] Among them, in this embodiment, a preset threshold is预先设置, and this preset threshold can be arbitrarily set according to requirements. For example, the preset threshold is 1 or 2, etc. The specific value of the preset threshold in this embodiment is not limited. Therefore, after the optical character recognition result is recognized in this embodiment, the number of characters in the optical character recognition result can be judged to determine whether the number of characters in the optical character recognition result is below the preset threshold. When it is determined that the number of characters in the optical character recognition result is below the preset threshold, the above sub-steps S1403 and S1404 are executed when performing translation conversion on the optical character recognition result based on the language display requirement.

[0261] In this embodiment, in order to address the problem of inaccurate translation that is likely to occur when there are few text recognition results corresponding to the first handwritten text, text extraction is performed on the third projection screen, and the obtained projection screen recognition result is used as a prompt word to translate the text recognition result, thereby obtaining an accurate translation result.

[0262] In combination with any of the above embodiments, this embodiment provides a handwritten text processing method. In this method, step S103 may include sub-steps S1501 and S1502:

[0263] S1501: Based on the language display requirement, the text recognition result is translated to obtain and display multiple candidate translation texts that meet the language display requirement.

[0264] In this embodiment, since the text recognition result corresponding to the first handwritten text identified has little content and is prone to inaccurate translation, when translating the text recognition result, the text recognition result is first translated based on the language display requirements to obtain and display multiple candidate translation texts that meet the language display requirements.

[0265] S1502: In response to a user's selection operation, determine the first translation text from the plurality of candidate translation texts.

[0266] In this embodiment, the user can determine the candidate translation text with the most accurate translation from the multiple candidate translation texts displayed, that is, in response to the user's selection operation, the candidate translation text selected by the user is determined from the multiple candidate translation texts, and the candidate translation text selected by the user is determined as the first translation text.

[0267] In other words, this embodiment addresses the problem of inaccurate translation that can occur when there are few text recognition results corresponding to the first handwritten text by presenting multiple translation results, which the user can select and confirm before displaying. For example, a translation result option is presented for each text recognition result, which is suspended above the default translation result and is numbered. The user clicks or manually enters the corresponding number on the keyboard to confirm the target translation result, i.e., the first translated text.

[0268] In combination with the above embodiments, in one implementation, when it is determined that the text recognition result corresponding to the first handwritten text includes a number of characters lower than a preset threshold, it can be determined first whether the collaborative editing permission has been obtained. When it is determined that the collaborative editing permission is possessed, text recognition is performed on the third projection screen to obtain a projection screen recognition result; through a natural language processing model, based on the language display requirements and the projection screen recognition result, the text recognition result is translated to obtain a first translation text that meets the language display requirements; when it is determined that the collaborative editing permission is not possessed, the text recognition result is translated based on the language display requirements to obtain and display multiple candidate translation texts that meet the language display requirements; in response to the user's selection operation, the first translation text is determined from the multiple candidate translation texts and displayed.

[0269] In combination with any of the above embodiments, this embodiment provides a handwritten text processing method. In this method, step S103 may specifically include sub-steps S1601 and S1602:

[0270] S1601: Based on the language display requirement, the text recognition result is translated into a converted intermediate text that meets the language display requirement.

[0271] In this embodiment, the text recognition result can be translated into a converted intermediate text that meets the language display requirements based on the set language display requirements. The converted intermediate text in this embodiment is a text that is only translated into a language without undergoing style conversion.

[0272] S1602: Perform text style conversion on the converted intermediate text character by character to obtain the first translated text.

[0273] In this embodiment, after obtaining the converted intermediate text, the text style conversion can be performed on the converted intermediate text character by character to obtain the first translated text. Specifically, the converted intermediate text can be beautified for each letter or character to obtain the beautified first translated text and display it.

[0274] In combination with any of the above embodiments, a handwritten text processing method of this embodiment may further include step S1701 in addition to the above steps, and step S103 may specifically include sub-steps S1702 and S1703:

[0275] S1701: Acquire target voice information synchronized with the first handwritten text.

[0276] The user may speak while handwriting input, such as a speaker writing while speaking or explaining the content while writing. Based on this, this embodiment can generate a first handwritten text in response to the user's handwriting operation and obtain target voice information synchronized with the first handwritten text. The target voice information in this embodiment is the voice mailbox collected in response to the user's handwriting operation.

[0277] In an optional embodiment, the time range for collecting the target voice information can be the same as the time range for responding to the user's handwriting operation, or it can be larger than the time range for responding to the user's handwriting operation. For example, the start time and / or end time of the user's handwriting operation is extended to serve as the time range for collecting the target voice information. This embodiment does not impose any restrictions on this.

[0278] S1702: Recognize the target voice information and obtain a voice recognition result.

[0279] In this embodiment, speech recognition can be performed on the acquired target speech information to obtain a speech recognition result corresponding to the target speech information. For example, the speech recognition result can be obtained by processing the target speech information through a speech recognition model.

[0280] S1703: Using a natural language processing model, based on the language presentation requirement and the speech recognition result, the text recognition result is translated to obtain the first translated text.

[0281] In this embodiment, the speech recognition results and text recognition results corresponding to the target speech information can be input into a pre-trained natural language processing model (for example: a natural language processing model based on prompt learning, such as GPT2, GPT3.5, GPT4, LLaMA or ChatGLM model, etc.). Through the pre-trained natural language processing model, the text recognition results are translated based on the language display requirements, and the speech recognition results corresponding to the target speech information are used as prompt words as translation prompts to obtain the first translated text that meets the language display requirements.

[0282] In this embodiment, in order to solve the problem of inaccurate translation that is less likely to occur in the text recognition results corresponding to the first handwritten text, voice recognition can be performed on the target voice information synchronized with the first handwritten text, and the obtained voice recognition results can be used as prompt words to perform translation prompts on the text recognition results, thereby obtaining accurate translation results.

[0283] In an optional embodiment, this embodiment adopts an adaptive method for selecting the default font for the translated content. For example, if the user of the handwriting device (such as the speaker) selects the beautification scheme of the Chinese recognition results as the regular script series (such as Songti, Weishu, Kaishu, etc.), the default font for handwriting beautification in other languages ​​is pre-set as the corresponding regular script form (English such as Times New Roman, Cambria, etc.) to perform style beautification on independent letters or characters. This can ensure the consistency of the beautification style of the handwriting beautification presented on the public screen page and the beautification style of its translated language.

[0284] In one embodiment, the handwriting device user can freely choose the presentation style of the translated content (i.e., the converted text that meets the language display requirements). In addition to using the corresponding standard font library font display, the user can also use intelligent handwriting text style generation to generate fonts with personal handwriting style to achieve a beautifying effect.

[0285] Among them, the font beautification algorithm of this embodiment takes the DG-Font network as an example. The DG-Font model example contains two branches, namely the content extraction branch and the style extraction branch. Content extraction branch: It is mainly composed of 3 layers of deformable convolution, 2 layers of residual blocks and a normalization layer. The convolution kernel step size used in the last two layers of the 3-layer deformable convolution is 2. Then for the 80*80 input image, it will be downsampled twice, that is, the encoded feature size is 20*20 (the output channel is set to 256). Style extraction branch: It is mainly composed of the vgg11 network, which includes a 3*3 ordinary convolution network and 5 layers of maximum pooling layers. In addition, an average pooling layer and a last fully connected layer are added, and the output dimension of the last fully connected layer is set to 128. That is, for an 80*80 image, after five layers of downsampling and pooling in VGG11, it becomes a 2*2 output, which is then converted to a 1*1 output (with 512 channels) through average pooling. The final fully connected layer then reduces the channels to 128 dimensions, outputting a feature vector of 1*1*128. Therefore, the style extraction branch encodes each image into a 128-dimensional feature vector. The decoder primarily consists of residual convolution blocks and upsampling layers. Since the content encoder converts the original 80*80 image to 20*20, the decoder uses two layers of upsampling to restore the 20*20 feature map to the original 80*80 size.

[0286] When used, this embodiment can be based on the handwritten text recognition algorithm. After detecting the second handwritten text, the single text is screenshotted as a standard 80*80 size image, input into the trained DG-Font network to obtain the beautified font, and the output result image is spliced ​​in the order of the original sentence to form the content with beautified font style, which is filled in the text box in the recognition area.

[0287] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0288] Based on the same inventive concept, an embodiment of the present invention further provides a method for processing handwritten text. Referring to FIG12 , a flow chart of a method for processing handwritten text according to this embodiment is shown. In this method, the method for processing handwritten text according to this embodiment may include the following steps:

[0289] S1801: Acquire a first handwritten text.

[0290] In this embodiment, during a meeting or teaching process, the user can directly import an image containing handwritten text into the device. Based on this, this embodiment can obtain a first handwritten text, which is an imported handwritten text image containing handwritten text.

[0291] S1802: Recognize the content of the first handwritten text and obtain a text recognition result.

[0292] S1803: Generate a first translation text of the first handwritten text based on language display requirements and the text recognition result.

[0293] S1804: Display the first translated text.

[0294] In this embodiment, steps S1802 to S1804 are the same or similar to the above-mentioned steps S102 and S104, and are not further described in this embodiment.

[0295] In this embodiment, for images containing handwritten text that are directly imported by the user during the meeting, this embodiment can obtain the first handwritten text, recognize the handwritten content, and translate and display the handwritten content according to the set language display requirements, so that participants who use other languages ​​can accurately understand the meaning of the written content through the reading of the receiving end, realize real-time and accurate translation and display of the handwritten content, and enhance the user's meeting experience.

[0296] It should be noted that, under the same inventive concept, any of the aforementioned embodiments can be combined with the embodiments under the inventive concept, which will not be described in detail.

[0297] Based on the same inventive concept, an embodiment of the present invention provides a handwritten text processing device 1300. Referring to FIG13 , FIG13 is a block diagram of the handwritten text processing device provided by an embodiment of the present invention. The device 1300 includes:

[0298] The text generation module 1301 is configured to generate a first handwritten text in response to a handwriting operation of a user;

[0299] A text recognition module 1302 is configured to recognize the content of the first handwritten text and obtain a text recognition result;

[0300] A text translation module 1303 is configured to generate a first translation text of the first handwritten text based on language display requirements and the text recognition result;

[0301] The text display module 1304 is configured to display the first translated text.

[0302] Optionally, the text translation module 1303 includes:

[0303] A first determining module is configured to determine a first text style of the first translated text, wherein the first text style includes at least one of the following: font, text color, and font size;

[0304] A first generating module is configured to generate a first translated text of the first handwritten text based on the language display requirement and the first text style.

[0305] Optionally, the first determining module includes:

[0306] The first display module is used to display multiple candidate text styles;

[0307] The second determining module is configured to determine, in response to a text style selection operation of the user, a candidate text style selected by the user as the first text style.

[0308] Optionally, the first determining module includes:

[0309] a third determining module, configured to determine a handwritten text style of the first handwritten text;

[0310] A fourth determining module is configured to determine a first text style of the first translated text based on the handwritten text style.

[0311] Optionally, the fourth determining module includes:

[0312] a fifth determining module, configured to match the handwritten text style with a plurality of standard text styles, and determine the standard text style with the highest matching degree as the first text style;

[0313] A sixth determining module is configured to determine the handwritten text style as the first text style.

[0314] Optionally, the apparatus 1300 is applied to a display system, the display system including a first device and a second device, the first device and the second device being communicatively connected; the apparatus 1300 further includes:

[0315] a second display module, deployed on the first device, configured to display the text recognition result according to requirements set on the second device;

[0316] The text translation module 1303 includes:

[0317] A text modification module is deployed on the first device and is used to modify the text recognition result into a first translation text that meets the language display requirements.

[0318] Optionally, the device 1300 is applied to a display system, the display system includes a screen projection sending end and a screen projection receiving end, the screen projection sending end and the screen projection receiving end are communicatively connected; the text display module 1304 includes:

[0319] A first sending module, deployed at the screen projection sending end, for sending the first screen projection image to the screen projection receiving end;

[0320] A third display module is deployed at the screen projection receiving end, and is used to display the first translated text on the first background map using the first screen projection image as the first background map.

[0321] Optionally, the apparatus 1300 is applied to a display system, the display system including a first device and a plurality of second devices, the first device and the plurality of second devices being communicatively connected; the apparatus 1300 further includes:

[0322] A second generating module, deployed on the plurality of second devices, for generating a plurality of first handwritten texts in response to a handwriting operation of a user;

[0323] A first recognition module, deployed on the plurality of second devices, configured to recognize the content of the first handwritten text generated by each of the second devices and obtain a plurality of text recognition results;

[0324] a second sending module, deployed on the plurality of second devices, for generating first translated texts of the plurality of first handwritten texts based on respective language display requirements and text recognition results, and sending the generated texts to the first devices;

[0325] A fourth display module is deployed on the first device and is used to display the first translated texts of the multiple first handwritten texts in a paginated manner.

[0326] Optionally, the fourth display module includes:

[0327] a tab determination module configured to, when the first device includes a large-screen terminal, determine a target tab from the plurality of tabs in response to a user's selection operation on the large-screen terminal;

[0328] The tab display module is configured to display a first translation text corresponding to the target tab based on the target tab.

[0329] Optionally, the apparatus 1300 further includes:

[0330] A control display module is deployed on the large-screen end and is used to control the mobile terminal to display the first translation text corresponding to the target tab when the first device also includes a mobile terminal communicatively connected to the large-screen end.

[0331] Optionally, the apparatus 1300 is applied to a display system, the display system including a first device and a second device, the first device being a handwriting device, the first device and the second device being communicatively connected; the text generation module 1301 including:

[0332] A first projection display module, deployed on the first device, for displaying a second projection screen;

[0333] A third generating module, deployed on the first device, is configured to generate a first handwritten text associated with the second projection screen in response to a handwriting operation of the user;

[0334] The apparatus 1300 further includes:

[0335] a third sending module, deployed on the first device, configured to send the second projection image and the first handwritten text to the second device;

[0336] The text display module 1304 includes:

[0337] A fifth display module is deployed on the second device and is used to display the second projection screen and the first translation text of the first handwritten text generated by the second device.

[0338] Optionally, the apparatus 1300 is applied to a display system, the display system including a first device and a second device, the first device and the second device being communicatively connected; the text generation module 1301 includes:

[0339] A second projection display module, deployed on the first device, for displaying a second projection screen;

[0340] a fourth generating module, deployed on the first device, configured to generate a first handwritten text associated with the second projection screen in response to a handwriting operation of a user;

[0341] The text translation module 1303 includes:

[0342] A first translation module, deployed on the first device, is configured to translate the text recognition result into a first translation text of the first handwritten text based on the language display requirement and the text recognition result.

[0343] The apparatus 1300 further includes:

[0344] a fourth sending module, deployed on the first device, configured to send the first translated text to the second device;

[0345] The text display module 1304 includes:

[0346] a sixth display module, deployed on the second device, configured to display the first translated text;

[0347] The conversion module is deployed on the second device and is used to re-convert the first translated text in response to a user's conversion operation and display the re-converted text.

[0348] Optionally, the apparatus 1300 further includes:

[0349] a seventh display module, deployed on the first device, configured to display the first translated text on the second background image using the second projection image as a second background image;

[0350] The fourth sending module includes:

[0351] The publishing response module is deployed on the first device and is used to send the first translated text to the second device in response to the publishing instruction.

[0352] Optionally, the text generation module 1301 includes:

[0353] The third projection display module is used to display the third projection screen;

[0354] a fifth generating module, configured to generate, in response to a handwriting operation of a user, a first handwritten text associated with the third projection screen;

[0355] The text translation module 1303 includes:

[0356] A projection screen recognition module, configured to perform text recognition on the third projection screen to obtain a projection screen recognition result;

[0357] The model processing module is used to translate the text recognition results through a natural language processing model based on the language display requirements and the projection screen recognition results to obtain the first translated text.

[0358] Optionally, the text translation module 1303 includes:

[0359] A translation candidate module is used to translate the text recognition results based on the language display requirements, obtain and display multiple candidate translation texts that meet the language display requirements;

[0360] The candidate selection module is configured to determine and display the first translation text from the plurality of candidate translation texts in response to a user's selection operation.

[0361] Optionally, the text translation module 1303 includes:

[0362] A translation conversion module, configured to translate the text recognition result into a converted intermediate text that meets the language display requirement based on the language display requirement;

[0363] A style conversion module is used to perform text style conversion on the converted intermediate text character by character to obtain the first translated text.

[0364] Optionally, the apparatus 1300 further includes:

[0365] A voice acquisition module, configured to acquire target voice information synchronized with the first handwritten text;

[0366] The text translation module 1303 includes:

[0367] A speech recognition module, configured to recognize the target speech information and obtain a speech recognition result;

[0368] The model recognition module is used to translate the text recognition result through a natural language processing model based on the language display requirements and the speech recognition result to obtain the first translated text.

[0369] Based on the same inventive concept, an embodiment of the present invention provides a handwritten text processing device 1400. Referring to FIG14 , FIG14 is a block diagram of the handwritten text processing device provided by an embodiment of the present invention. The device 1400 includes:

[0370] A text acquisition module 1401 is used to acquire a first handwritten text;

[0371] A content recognition module 1402 is configured to recognize the content of the first handwritten text and obtain a text recognition result;

[0372] A text conversion module 1403 is configured to generate a first translation text of the first handwritten text based on language display requirements and the text recognition result;

[0373] The text display module 1404 is configured to display the first translated text.

[0374] Based on the same inventive concept, another embodiment of the present invention provides a display system, which includes: a first device and a second device, wherein the first device is communicatively connected to the second device; when the first device and / or the second device is executed, the steps of the handwritten text processing method as described in any of the above embodiments of the present invention are implemented.

[0375] Based on the same inventive concept, another embodiment of the present invention provides an electronic device 1500, as shown in Figure 15. Figure 15 is a schematic diagram of an electronic device according to one embodiment of the present invention. The electronic device includes a processor 1501, a memory 1502, and a computer program stored in the memory 1502 and executable on the processor 1501. When executed by the processor, the computer program implements the steps of the handwritten text processing method described in any of the above embodiments of the present invention.

[0376] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the handwritten text processing method described in any of the above embodiments of the present invention are implemented.

[0377] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0378] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, apparatuses, or computer program products. Therefore, the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0379] This embodiment is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to this embodiment. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as combinations of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable image processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable image processing terminal device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more blocks in the block diagram.

[0380] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable image processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0381] These computer program instructions can also be loaded onto a computer or other programmable image processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0382] Although the preferred embodiment of the present embodiment has been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the present embodiment.

[0383] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0384] The above is a detailed introduction to the handwritten text processing method, device, display system, equipment and medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A handwritten text processing method, characterized in that, The method includes: Generating first handwritten text in response to a user's handwriting operation; Identifying the content of the first handwritten text to obtain a text recognition result; Generating a first translation text of the first handwritten text based on the language display requirement and the text recognition result; Displaying the first translation text.

2. The method according to claim 1, characterized in that, The generating the first translation text of the first handwritten text based on the language display requirement and the text recognition result includes: Determining a first text style of the first translation text, where the first text style includes at least one of the following: font, text color, font size; Generating the first translation text of the first handwritten text based on the language display requirement and the first text style.

3. The method according to claim 2, wherein The determining the first text style of the first translation text includes: Displaying a plurality of candidate text styles; In response to a user's text style selection operation, determining one of the candidate text styles selected by the user as the first text style.

4. The method according to claim 2, wherein The determining the first text style of the first translation text includes: Determining the handwritten text style of the first handwritten text; Based on the handwritten text style, determining the first text style of the first translation text.

5. The method according to claim 4, wherein The determining the first text style of the first translation text based on the handwritten text style includes: Matching the handwritten text style with a plurality of standard text styles, and determining the standard text style with the highest matching degree as the first text style; or Determining the handwritten text style as the first text style.

6. The method according to claim 1, wherein The method is applied to a display system, where the display system includes a first device and a second device, and the first device and the second device are communicatively connected; the method further includes: The first device displays the text recognition result according to the requirement set by the second device; The generating the first translation text of the first handwritten text based on the language display requirement and the text recognition result includes: The first device changes the text recognition result to a first translation text that meets the language display requirement.

7. The method according to claim 1, characterized in that, The method is applied to a display system, where the display system includes a screen mirroring sending end and a screen mirroring receiving end, and the screen mirroring sending end and the screen mirroring receiving end are communicatively connected; The displaying the first translation text includes: The screen mirroring sending end sends a first screen mirroring image to the screen mirroring receiving end; The screen mirroring receiving end uses the first screen mirroring image as a first background image and displays the first translation text on the first background image.

8. The method according to claim 1, wherein The method is applied to a display system, where the display system includes a first device and a plurality of second devices, and the first device and the plurality of second devices are communicatively connected; the method further includes: The plurality of second devices generate a plurality of first handwritten texts in response to a user's handwriting operation; The plurality of second devices identify the content of the first handwritten texts generated by themselves to obtain a plurality of text recognition results; The plurality of second devices generate first translation texts of the plurality of first handwritten texts based on their respective language display requirements and text recognition results and send them to the first device; The first device performs a paged tab display of the first translation texts of the plurality of first handwritten texts.

9. The method according to claim 8, characterized in that, The first device performs paged tab display of the first translation texts of the multiple first handwritten texts, including: When the first device includes a large screen terminal, in response to a user's selection operation on the large screen terminal, a target tab is determined among the multiple tabs; Based on the target tab, the first translation text corresponding to the target tab is displayed.

10. The method according to claim 9, wherein The method further includes: When the first device further includes a mobile terminal communicatively connected to the large screen terminal, the large screen terminal controls the mobile terminal to display the first translation text corresponding to the target tab.

11. The method according to claim 1, characterized in that, The method is applied to a display system, the display system includes a first device and a second device, the first device is a handwriting device, and the first device and the second device are communicatively connected; The generating of the first handwritten text in response to a user's handwriting operation includes: The first device displays a second screen mirroring image; The first device, in response to a user's handwriting operation, generates a first handwritten text associated with the second screen mirroring image; The method further includes: The first device sends the second screen mirroring image and the first handwritten text to the second device; The displaying of the first translation text includes: The second device displays the second screen mirroring image and the first translation text of the first handwritten text generated by the second device.

12. The method according to claim 1, wherein The method is applied to a display system, the display system includes a first device and a second device, and the first device and the second device are communicatively connected; The generating of the first handwritten text in response to a user's handwriting operation includes: The first device displays a second screen mirroring image; The first device, in response to a user's handwriting operation, generates a first handwritten text associated with the second screen mirroring image; The generating of the first translation text of the first handwritten text based on the language display requirement and the text recognition result includes: The first device translates the text recognition result into the first translation text of the first handwritten text based on the language display requirement and the text recognition result; The method further includes: The first device sends the first translation text to the second device; The displaying of the first translation text includes: The second device displays the first translation text; The second device, in response to a user's conversion operation, re-converts the first translation text and displays the re-converted text.

13. The method according to claim 12, wherein The method further includes: The first device uses the second screen mirroring image as a second background image and displays The first translation text; The first device sending the first translation text to the second device includes: The first device, in response to a publishing instruction, sends the first translation text to the second device.

14. The method according to claim 1, wherein The generating of the first handwritten text in response to a user's handwriting operation includes: Displaying a third screen mirroring image; In response to a user's handwriting operation, generating a first handwritten text associated with the third screen mirroring image; The generating of the first translation text of the first handwritten text based on the language display requirement and the text recognition result includes: Performing character recognition on the third screen mirroring image to obtain a screen mirroring image recognition result; Through a natural language processing model, based on the language display requirement and the recognition result of the screen projection image, the text recognition result is translated to obtain the first translated text.

15. The method according to claim 1, wherein Generating the first translated text of the first handwritten text based on the language display requirement and the text recognition result includes: Based on the language display requirement, the text recognition result is translated to obtain multiple candidate translated texts that meet the language display requirement and are displayed; In response to the user's selection operation, the first translated text is determined among the multiple candidate translated texts.

16. The method according to claim 1, characterized in that, Generating the first translated text of the first handwritten text based on the language display requirement and the text recognition result includes: Based on the language display requirement, the text recognition result is translated into a converted intermediate text that meets the language display requirement; The text style of the converted intermediate text is converted character by character to obtain the first translated text.

17. The method according to claim 1, wherein The method further includes: Obtaining target voice information synchronized with the first handwritten text; Generating the first translated text of the first handwritten text based on the language display requirement and the text recognition result includes: Recognizing the target voice information to obtain a voice recognition result; Through a natural language processing model, based on the language display requirement and the voice recognition result, the text recognition result is translated to obtain the first translated text.

18. A method for processing handwritten text, characterized in that, The method includes: Obtaining a first handwritten text; Recognizing the content of the first handwritten text to obtain a text recognition result; Generating the first translated text of the first handwritten text based on the language display requirement and the text recognition result; Displaying the first translated text.

19. A handwritten text processing device, characterized in that, The device includes: A text generation module, configured to generate a first handwritten text in response to a user's handwriting operation; A text recognition module, configured to recognize the content of the first handwritten text to obtain a text recognition result; A text translation module, configured to generate the first translated text of the first handwritten text based on the language display requirement and the text recognition result; A text display module, configured to display the first translated text.

20. A handwritten text processing device, characterized in that, The device includes: A text acquisition module, configured to acquire a first handwritten text; A content recognition module, configured to recognize the content of the first handwritten text to obtain a text recognition result; A text conversion module, configured to generate the first translated text of the first handwritten text based on the language display requirement and the text recognition result; A text display module, configured to display the first translated text.

21. A display system, characterized in that, The display system includes: a first device and a second device, the first device is communicatively connected to the second device; when the first device and / or the second device execute, the steps of the handwritten text processing method according to any one of claims 1 to 18 are implemented.

22. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the handwritten text processing method according to any one of claims 1 to 18 are implemented.

23. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the handwritten text processing method according to any one of claims 1 to 18 are implemented.

Images