Text translation method, device, scanning pen and storage medium

Abstract

The present invention provides a text translation method, device, scanning pen and storage medium, the method comprising: obtaining a sentence text to be translated, and a regional text of a region where the sentence text is located; performing text translation on the regional text to obtain a regional text translation result; and determining a translation result of the sentence text from the regional text translation result based on the position information of the sentence text in the regional text. The text translation method, device, scanning pen and storage medium provided by the present invention can obtain a regional text translation result that fits the meaning of the regional text by performing text translation on the regional text of the region where the sentence text is located, and determine a translation result of the sentence text from the regional text translation result based on the position information of the sentence text in the regional text, so that the translation result of the sentence text can fit the meaning of the regional text, thereby improving the text translation accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text translation method, device, scanning pen and storage medium. Background Art

[0002] Image-to-text translation is the process of automatically translating the source language contained in an image or video into a target language using a computer system. Image-to-text translation technology can help people translate and understand the text content in images and videos quickly and effectively.

[0003] At present, the image of the text to be translated is obtained by scanning the device, and then the text is recognized by the image to obtain the text to be translated, and the text to be translated is translated to obtain the translation result. However, the above translation result is usually out of the context of the article where the text to be translated is located, resulting in low accuracy of the translation result. Summary of the invention

[0004] The present invention provides a text translation method, device, scanning pen and storage medium, which are used to solve the defect of low text translation accuracy in the prior art.

[0005] The present invention provides a text translation method, comprising:

[0006] Obtaining a sentence text to be translated and a region text of a region where the sentence text is located;

[0007] Performing text translation on the regional text to obtain a regional text translation result;

[0008] Based on the position information of the sentence text in the region text, the translation result of the sentence text is determined from the region text translation results.

[0009] According to a text translation method provided by the present invention, the step of obtaining a sentence text to be translated includes:

[0010] Acquire a sentence image corresponding to the sentence text and a region image of the region where the sentence text is located;

[0011] fusing the sentence image and the region image, and updating the sentence image based on the fusion result;

[0012] Perform text recognition on the updated sentence image to obtain the sentence text.

[0013] According to a text translation method provided by the present invention, the sentence image is collected by a first scanning element, and the first scanning element is located at a scanning end of a scanning pen; the area image is collected by a second scanning element, and the second scanning element is located at an end away from the scanning end.

[0014] According to a text translation method provided by the present invention, the position information is determined based on the following steps:

[0015] Perform sentence processing on the regional text to obtain sentences corresponding to the regional text;

[0016] The sentence text is matched with each clause, and the position information is determined based on the position of the clause matched with the sentence text.

[0017] According to a text translation method provided by the present invention, the translation result of the sentence text is determined from the regional text translation result, and then the method further includes:

[0018] Determine the speaker role of the sentence text based on the position information and the area text;

[0019] Based on the speaker role of the sentence text, the translation result of the sentence text and the sentence text are subjected to speech synthesis to obtain synthesized speech and play it.

[0020] The present invention also provides a text translation device, comprising:

[0021] A text acquisition unit, used to acquire a sentence text to be translated and a regional text of a region where the sentence text is located;

[0022] A regional text translation unit, used for text translation of the regional text to obtain a regional text translation result;

[0023] The sentence text translation unit is used to determine the translation result of the sentence text from the regional text translation results based on the position information of the sentence text in the regional text.

[0024] The present invention also provides a scanning pen, comprising: a first scanning element, a second scanning element, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the text translation method as described above when executing the program, wherein the first scanning element is located at a scanning end and is used to acquire a sentence image corresponding to the sentence text, and the second scanning element is located at an end away from the scanning end and is used to acquire an area image of the area text.

[0025] A scanning pen provided according to the present invention further includes: a first switch, which is connected to the first scanning element and the second scanning element respectively, and is used to control the first scanning element to capture the sentence image, and control the second scanning element to capture the area image.

[0026] A scanning pen provided according to the present invention further includes: a second switch, wherein the second switch is connected to the second scanning element and is used to control the second scanning element to capture a picture book image so that the processor performs picture book recognition based on the picture book image.

[0027] A scanning pen provided according to the present invention further includes: a display unit, used to display the translation result of the sentence text, or the translation result of the sentence text and the translation result of the area text.

[0028] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned text translation methods are implemented.

[0029] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned text translation methods are implemented.

[0030] The text translation method, device, scanning pen and storage medium provided by the present invention can obtain a regional text translation result that fits the meaning of the regional text by performing text translation on the regional text of the region where the sentence text is located, and determine the translation result of the sentence text from the regional text translation result based on the position information of the sentence text in the regional text, so that the translation result of the sentence text can fit the meaning of the regional text, thereby improving the text translation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a flowchart of the text translation method provided by the present invention;

[0033] Figure 2 is a flowchart of an implementation of step 110 in the text translation method provided by the present invention;

[0034] Figure 3 It is a flow chart of a method for obtaining position information in a text translation method provided by the present invention;

[0035] Figure 4 It is a flow chart of the voice playing method in the text translation method provided by the present invention;

[0036] Figure 5is a structural schematic diagram of a text translation device provided by the present invention;

[0037] Figure 6 This is one of the structural schematic diagrams of the scanning pen provided by the present invention;

[0038] Figure 7 This is the second structural schematic diagram of the scanning pen provided by the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] At present, when translating text, the image of the text to be translated is obtained by scanning the device, and then optical character recognition (OCR) is performed on the image to obtain the text to be translated, and the text to be translated is translated (such as Chinese-English translation) to obtain the translation result. However, the text to be translated is usually a sentence in an article, and the above translation result is obtained by translating the article where the text to be translated is located, so that the translation result is out of touch with the meaning of the article, which reduces the accuracy of the translation result.

[0041] To this end, the present invention provides a text translation method. Figure 1 It is a flowchart of the text translation method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0042] Step 110: Obtain the sentence text to be translated and the area text of the area where the sentence text is located.

[0043] Specifically, the sentence text is the sentence text that needs to be translated, and the regional text is the text of the region where the sentence text is located. That is, it can be understood that the regional text contains the sentence text and the context text of the sentence text. For example, the regional text can be a paragraph text of the paragraph where the sentence text is located, or it can be the entire article text of the article where the sentence text is located.

[0044] The sentence text and area text here can be directly input by the user, or obtained by transcribing the collected audio, or obtained by collecting images through image acquisition devices such as scanners, mobile phones, cameras, and performing OCR text recognition on the images. The embodiments of the present invention do not make specific limitations on this.

[0045] Step 120: Perform text translation on the regional text to obtain a regional text translation result.

[0046] Specifically, after obtaining the regional text, the regional text can be input into the text translation model, and the regional text can be translated by the text translation model to obtain the regional text translation result. Since the regional text translation result is obtained by translating the regional text of the region where the sentence text is located, it can be understood that the regional text translation result is obtained by translating according to the text content of the regional text, so that the obtained regional text translation result can accurately fit the meaning of the regional text of the region where the sentence text is located.

[0047] Optionally, before translating the regional text, a text translation model can be pre-trained, which can be achieved by performing the following steps: first, a large number of sample texts are collected, and the corresponding sample text translation results are determined by manual annotation. Then, the initial model is trained based on the sample texts and their corresponding sample text translation results to obtain a text translation model.

[0048] Step 130: Determine the translation result of the sentence text from the regional text translation results based on the position information of the sentence text in the regional text.

[0049] Specifically, since the regional text contains not only the sentence text but also the context text of the sentence text, the regional text translation result contains both the translation result of the sentence text and the translation result of the context text of the sentence text. Based on the position information of the sentence text in the regional text, the position of the translation result of the sentence text in the regional text translation result can be determined, so that the translation result of the sentence text can be extracted from the regional text translation result. Among them, the position information of the sentence text in the regional text can be determined by a pre-marked label, or it can be obtained by processing the regional text into sentences and matching each sentence with the sentence text. The embodiment of the present invention does not specifically limit this.

[0050] In addition, the regional text translation result is obtained by translating the text content of the regional text, so that the regional text translation result can accurately fit the meaning of the regional text, that is, the translation result of the sentence text contained in the regional text translation result can also accurately fit the meaning of the regional text. Compared with the traditional method of translating the sentence text to be translated separately from the meaning of the paragraph or article where the sentence text is located, the method provided by the embodiment of the present invention can determine the translation result of the sentence text based on the meaning of the regional text, thereby improving the accuracy and fit of the text translation, and thus ensuring the reading fluency of the translation result.

[0051] The text translation method provided by the embodiment of the present invention performs text translation on the regional text of the region where the sentence text is located, so as to obtain a regional text translation result that fits the meaning of the regional text, and based on the position information of the sentence text in the regional text, determines the translation result of the sentence text from the regional text translation result, so that the translation result of the sentence text can fit the meaning of the regional text, thereby improving the text translation accuracy.

[0052] The following is a specific example to illustrate the process of determining the translation result of a sentence text:

[0053] The sentence text is:

[0054] The news reported that Chinese tutoring market has great potential, because the parents are willing to pay more money on their children's education.

[0055] The area text is:

[0056] (a) Many teachers gain their income by giving students extracurricular tutoring, which can earn much more than their salary. (b) The news reported that Chinese tutoring market has great potential, because the parents are willing to pay more money on their children's education. (c) It seems that being ateacher means earning more easily.

[0057] As can be seen from the above, the regional text is the paragraph text of the region where the sentence text is located, which includes not only the sentence text - sentence b, but also the context texts of the sentence text, sentence a and sentence c. After determining the sentence text and the regional text, the regional text can be translated to obtain the regional text translation result, which includes the translation results corresponding to sentences a, b and c, and the translation result of sentence b is obtained by combining the context (sentence a and sentence c) of the translation, so that the translation result of sentence b can fit the meaning of the regional text. Therefore, taking the translation result of sentence b as the translation result of the sentence text can accurately fit the meaning of the regional text and improve the accuracy of text translation.

[0058] Based on the above embodiments, Figure 2 1 is a flow chart of an implementation of step 110 in the text translation method provided by the present invention. The step 110 of obtaining the sentence text to be translated includes:

[0059] Step 111: Obtain a sentence image corresponding to the sentence text and a region image of the region where the sentence text is located;

[0060] Step 112: Fusing the sentence image and the region image, and updating the sentence image based on the fusion result;

[0061] Step 113: Perform text recognition on the updated sentence image to obtain the sentence text.

[0062] Specifically, the sentence image refers to the image corresponding to the sentence text, and the region image refers to the image corresponding to the region text. The sentence image contains image information of the sentence text, and the region image contains image information of the sentence text and image information of the context of the sentence text, that is, both the sentence image and the region image contain image information of the sentence text.

[0063] Among them, sentence images and area images can be acquired through image acquisition devices such as scanners, mobile phones, and cameras. Since the image acquisition devices themselves may have errors, the image information of the sentence text contained in the acquired sentence images is inaccurate. If text recognition is performed directly based on the sentence images acquired by the image acquisition device, inaccurate sentence text may be obtained.

[0064] Therefore, the embodiment of the present invention fuses the sentence image and the region image, so as to perform image fusion on the image information of the sentence text contained in both the sentence image and the region image, thereby obtaining a fused image that can accurately represent the image information of the sentence text, and then updating the original sentence image based on the image information of the sentence text in the fused image, so that the updated sentence image can accurately represent the image information of the sentence text.

[0065] After obtaining the updated sentence image, text recognition is performed on the updated sentence image, so that the sentence text can be accurately obtained.

[0066] The text translation method provided by the embodiment of the present invention fuses the sentence image and the region image and updates the sentence image based on the fusion result, so that the sentence text can be accurately obtained based on the updated sentence image.

[0067] Based on any of the above embodiments, the sentence image is collected by the first scanning element, and the first scanning element is located at the scanning end of the scanning pen; the area image is collected by the second scanning element, and the second scanning element is located at the end away from the scanning end.

[0068] Specifically, the sentence image is acquired by the first scanning element on the scanning pen, and the area image is acquired by the second scanning element on the scanning pen. The first scanning element is located at the scanning end of the scanning pen, and the second scanning element is located at the end away from the scanning pen, such as the first scanning element can be located at the tip of the scanning pen, and the second scanning element can be located at the other end of the tip of the scanning pen.

[0069] Optionally, the first scanning element and the second scanning element can be controlled by the same switch, such as a switch can be set at the tip of the scanning pen, and when the switch is pressed, the first scanning element and the second scanning element are both turned on to respectively acquire the sentence image and the area image. The first scanning element and the second scanning element can also be controlled by different switches, for example, a switch for controlling the first scanning element can be set at the tip of the scanning pen, and a switch for controlling the second scanning element can be set at the other end of the tip of the scanning pen.

[0070] In addition, since the second scanning element is used to capture images of a larger area, in order to ensure the integrity and accuracy of the data captured by the second scanning element, the second scanning element may be an AI camera with an automatic tracking function.

[0071] Based on any of the above embodiments, Figure 3 is a flow chart of a method for obtaining position information in a text translation method provided by the present invention, such as Figure 3 As shown, the location information is determined based on the following steps:

[0072] Step 310: Sentence processing is performed on the regional text to obtain the sentences corresponding to the regional text;

[0073] Step 320: Match the sentence text with each clause, and determine position information based on the position of the clause that matches the sentence text.

[0074] Specifically, after obtaining the regional text, the regional text is segmented into multiple sentences, and each sentence carries its position information in the regional text. When segmenting the regional text, the regional text can be segmented according to the punctuation marks at the end of the sentence to obtain each sentence.

[0075] After obtaining each sentence of the regional text, the sentence text is matched with each sentence to determine the sentence matching the sentence text, and the sentence position of the corresponding sentence is used as position information, so that the translation result of the sentence text can be determined from the regional text translation result according to the position information.

[0076] It can be seen that the embodiment of the present invention matches the sentence text with each sentence of the regional text, so that the position information can be accurately determined based on the sentence position matching the sentence text, and then the translation result of the sentence text can be accurately determined from the regional text translation result based on the position information.

[0077] Based on any of the above embodiments, Figure 4 FIG. 1 is a flow chart of a voice playing method in a text translation method provided by the present invention. Figure 4 As shown, the translation result of the sentence text is determined from the regional text translation result, and then it also includes:

[0078] Step 410: Determine the speaker role of the sentence text based on the position information and the region text;

[0079] Step 420: Based on the speaker role of the sentence text, the translation result of the sentence text and the sentence text are synthesized into speech to obtain the synthesized speech and play it.

[0080] Specifically, the region text may be a dialogue scene text, and the corresponding sentence text may be a sentence of a speaker in the dialogue scene text. Since the region text contains speaker role information corresponding to each sentence, the speaker role of the sentence text may be determined from the region text based on the position information.

[0081] After the speaker role of the sentence text is determined, speech synthesis is performed on the translation result of the sentence text and the sentence text according to the corresponding speaker role, so that synthesized speech can be obtained and played.

[0082] It can be seen that the embodiment of the present invention determines the speaker role of the sentence text based on the position information and the regional text, so that the translation result of the sentence text and the sentence text can be synthesized based on the speaker role of the sentence text to obtain the synthesized speech for representing the original dialogue scene and play it.

[0083] The following is a specific example to illustrate the process of determining the synthesized speech:

[0084] The sentence text is:

[0085] What is in your schoolbag?

[0086] The area text is:

[0087] A:Look! It is black and white.

[0088] B:Wow! It is a panda! What is in your schoolbag?

[0089] A:An English book,a maths book,three storybooks and…

[0090] B:It is a fat panda!

[0091] From the above, we can see that the regional text contains a dialogue scene text containing two speakers, A and B. Based on the position information of the sentence text "What is in your schoolbag?" in the regional text, it can be determined that the speaker role of the sentence text is B. Therefore, based on the speaker role B, the sentence text "What is in your schoolbag?" and its corresponding translation result can be speech synthesized to obtain the synthesized speech and play it, that is, the synthesized speech is played in B's voice, thereby restoring the voice characteristics of the characters in the dialogue scene to improve the user experience.

[0092] Based on any of the above embodiments, the present invention further provides a text translation method, the method comprising:

[0093] First, the sentence image corresponding to the sentence text and the region image of the region where the sentence text is located are obtained. Then, the sentence image and the region image are fused, and the sentence image is updated based on the fusion result.

[0094] After obtaining the updated sentence image, text recognition is performed on the sentence image to obtain the sentence text, and text recognition is performed on the region image to obtain the region text. Text translation is performed on the region text to obtain a region text translation result, and based on the position information of the sentence text in the region text, the translation result of the sentence text is determined from the region text translation result.

[0095] In addition, when the regional text is a dialogue scene text and the sentence text is a sentence of a speaker in the dialogue scene, the speaker role corresponding to the sentence text is determined from the regional text based on the position information of the sentence text in the regional text, and the translation result of the sentence text and the sentence text are speech synthesized based on the speaker role to obtain the synthesized speech and play it.

[0096] It can be seen that the text translation method provided by the embodiment of the present invention performs text translation on the regional text of the region where the sentence text is located, so as to obtain a regional text translation result that fits the meaning of the regional text, and based on the position information of the sentence text in the regional text, determines the translation result of the sentence text from the regional text translation result, so that the translation result of the sentence text can fit the meaning of the regional text, thereby improving the text translation accuracy.

[0097] The text translation device provided by the present invention is described below. The text translation device described below and the text translation method described above can be referred to each other.

[0098] Based on any of the above embodiments, Figure 5 is a schematic diagram of the structure of the text translation device provided by the present invention, such as Figure 5 As shown, the device comprises:

[0099] The text acquisition unit 510 is used to acquire the sentence text to be translated and the regional text of the region where the sentence text is located;

[0100] A regional text translation unit 520 is used to perform text translation on the regional text to obtain a regional text translation result;

[0101] The sentence text translation unit 530 is used to determine the translation result of the sentence text from the regional text translation results based on the position information of the sentence text in the regional text.

[0102] Based on any of the above embodiments, the text acquisition unit 510 includes:

[0103] A text image acquisition unit, used to acquire a sentence image corresponding to the sentence text, and an area image of an area where the sentence text is located;

[0104] an image fusion unit, configured to fuse the sentence image and the region image, and update the sentence image based on a fusion result;

[0105] The text recognition unit is used to perform text recognition on the updated sentence image to obtain the sentence text.

[0106] Based on any of the above embodiments, the sentence image is collected by a first scanning element, which is located at a scanning end of the scanning pen; the area image is collected by a second scanning element, which is located at an end away from the scanning end.

[0107] Based on any of the above embodiments, the device further includes:

[0108] A sentence division unit, used for performing sentence division processing on the regional text to obtain each sentence corresponding to the regional text;

[0109] The matching unit is used to match the sentence text with each clause, and determine the position information based on the position of the clause matching the sentence text.

[0110] Based on any of the above embodiments, the device further includes:

[0111] a role determination unit, configured to determine the speaker role of the sentence text based on the position information and the region text after determining the translation result of the sentence text from the region text translation result;

[0112] The speech synthesis unit is used to perform speech synthesis on the translation result of the sentence text and the sentence text based on the speaker role of the sentence text, obtain the synthesized speech and play it.

[0113] Figure 6 It is one of the structural schematic diagrams of the scanning pen provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610, a memory 620, a communication interface 630, a communication bus 640, a first scanning element 650 arranged at a scanning end, and a second scanning element 660 arranged at an end away from the scanning end, wherein the first scanning element 650 is used to collect sentence images corresponding to sentence texts, and the second scanning element 660 is used to collect regional images of regional texts, and the processor 610, the memory 620, the communication interface 630, the first scanning element 650 and the second scanning element 660 complete mutual communication through the communication bus 640. The processor 610 can call the logic instructions in the memory 620 to execute the text translation method, which includes: obtaining a sentence text to be translated, and a regional text of the region where the sentence text is located; performing text translation on the regional text to obtain a regional text translation result; and determining the translation result of the sentence text from the regional text translation result based on the position information of the sentence text in the regional text.

[0114] In addition, the logic instructions in the above-mentioned memory 620 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0115] Based on any of the above embodiments, Figure 7 This is the second structural schematic diagram of the scanning pen provided by the present invention. Figure 7 As shown, the scanning pen includes a first scanning element 650, a second scanning element 660 and a first switch 670. The first switch 670 is connected to the first scanning element 650 and the second scanning element 660 respectively, and is used to control the first scanning element 650 to capture sentence images, and control the second scanning element 660 to capture area images.

[0116] Optionally, the first switch 670 can be a mechanical trigger switch, and the first switch 670 can be pressed as the scanning pen body, at which time the first scanning element 650 starts to collect sentence images, and the second scanning element 660 starts to collect area images. When the scanning pen body is lifted, the first switch 670 automatically resets, and the first scanning element 650 and the second scanning element 660 are closed, so as to control the first scanning element 650 and the second scanning element 660 to start and end scanning as the scanning action starts and ends, thereby improving the experience of the scanning pen application. In order to ensure the integrity and accuracy of the data collected by the second scanning element 660, the second scanning element 660 can use an AI camera with automatic tracking function.

[0117] Based on any of the above embodiments, the scanning pen further includes: a second switch, wherein the second switch is connected to the second scanning element 660, and is used to control the second scanning element 660 to collect the picture book image, so that the processor 610 can perform picture book recognition based on the picture book image. In addition, the second switch can be a button switch located on the body of the scanning pen, or a touch switch located in the display unit 680, which is not specifically limited in the embodiment of the present invention.

[0118] When the second switch is turned on, the picture book recognition function of the second scanning element 660 is turned on, and other functions of the second scanning element 660 are turned off, so that the second scanning element 660 can be used to capture the picture book image, so that the processor 610 can perform picture book recognition based on the picture book image; when the second switch is turned off, the picture book recognition function of the second scanning element 660 is turned off, and other functions of the second scanning element 660 are turned on.

[0119] Based on any of the above embodiments, Figure 7 As shown, the scanning pen further includes: a display unit 680, which is used to display the translation result of the sentence text, or the translation result of the sentence text and the translation result of the area text.

[0120] Optionally, the display unit 680 may include a display screen for displaying the translation result of the sentence text, or the translation result of the sentence text and the translation result of the regional text. The display unit 680 may also include a playback module, which plays the corresponding synthesized speech in the speaker role of the sentence text when the regional text is a dialogue scene text.

[0121] It is understandable that the processor 110 may also be provided with a translation module for receiving the sentence image captured by the first scanning element 650 and the area image captured by the second scanning element 660, and obtaining the corresponding sentence text and area text based on the sentence image and the area image, and then performing text translation on the area text to obtain the area text translation result, and based on the position of the sentence text in the translated text, determining the translation result of the sentence text from the area text translation result, and displaying it through the display unit 680.

[0122] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the text translation method provided by the above methods, and the method includes: obtaining a sentence text to be translated, and a regional text of the area where the sentence text is located; performing text translation on the regional text to obtain a regional text translation result; and determining the translation result of the sentence text from the regional text translation result based on the position information of the sentence text in the regional text.

[0123] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned text translation methods, the methods comprising: obtaining a sentence text to be translated, and a regional text of a region where the sentence text is located; performing text translation on the regional text to obtain a regional text translation result; and determining a translation result of the sentence text from the regional text translation result based on the position information of the sentence text in the regional text.

[0124] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text translation method, characterized in that: include: Obtaining a sentence text to be translated and a region text of a region where the sentence text is located; Performing text translation on the regional text to obtain a regional text translation result; Determining a translation result of the sentence text from the regional text translation results based on the position information of the sentence text in the regional text; The step of obtaining the sentence text to be translated includes: Acquire a sentence image corresponding to the sentence text and a region image of the region where the sentence text is located; fusing the sentence image and the region image, and updating the sentence image based on the fusion result; Perform text recognition on the updated sentence image to obtain the sentence text.

2. The text translation method according to claim 1, characterized in that: The sentence image is collected by a first scanning element, which is located at a scanning end of the scanning pen; the area image is collected by a second scanning element, which is located at an end away from the scanning end.

3. The text translation method according to any one of claims 1 to 2, characterized in that: The location information is determined based on the following steps: Perform sentence processing on the regional text to obtain sentences corresponding to the regional text; The sentence text is matched with each clause, and the position information is determined based on the position of the clause matched with the sentence text.

4. The text translation method according to any one of claims 1 to 2, characterized in that: The step of determining the translation result of the sentence text from the regional text translation result further includes: Determine the speaker role of the sentence text based on the position information and the area text; Based on the speaker role of the sentence text, the translation result of the sentence text and the sentence text are subjected to speech synthesis to obtain synthesized speech and play it.

5. A text translation device, characterized in that: include: A text acquisition unit, used to acquire a sentence text to be translated and a regional text of a region where the sentence text is located; A regional text translation unit, used for text translation of the regional text to obtain a regional text translation result; A sentence text translation unit, configured to determine a translation result of the sentence text from the regional text translation results based on position information of the sentence text in the regional text; The step of obtaining the sentence text to be translated includes: Acquire a sentence image corresponding to the sentence text and a region image of the region where the sentence text is located; fusing the sentence image and the region image, and updating the sentence image based on the fusion result; Perform text recognition on the updated sentence image to obtain the sentence text.

6. A scanning pen, characterized in that: include: A first scanning element, a second scanning element, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the text translation method according to any one of claims 1 to 4 are implemented, wherein the first scanning element is located at a scanning end and is used to acquire a sentence image corresponding to the sentence text, and the second scanning element is located at an end away from the scanning end and is used to acquire a regional image of the regional text.

7. The scanning pen according to claim 6, characterized in that: Also includes: A first switch, wherein the first switch is connected to the first scanning element and the second scanning element respectively, and is used to control the first scanning element to collect the sentence image, and control the second scanning element to collect the area image.

8. The scanning pen according to claim 6, characterized in that: Also includes: A second switch, wherein the second switch is connected to the second scanning element and is used to control the second scanning element to capture a picture book image, so that the processor performs picture book recognition based on the picture book image.

9. The scanning pen according to claim 6, characterized in that: Also includes: The display unit is used to display the translation result of the sentence text, or the translation result of the sentence text and the translation result of the region text.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the text translation method according to any one of claims 1 to 4 are implemented.

Images