Translation method and device, equipment and medium

Abstract

The invention provides a translation method and device, equipment and a medium, and relates to the technical field of language processing.The method comprises the steps that a target text image is recognized, and text content of a to-be-translated text in the target text image and book representation features of the to-be-translated text are obtained; determining a book type of the book and context information of the to-be-translated text in the book according to the book characterization feature; and determining translation display information of the to-be-translated text according to the text content, the book type and the context information. The to-be-translated text is translated in an aided mode according to the context information in the book where the to-be-translated text is located and the book type, the integrating degree of the translation display information of the to-be-translated text and the translation style and the context of the book where the to-be-translated text is located can be effectively improved, and the difference between the translation content and the context is reduced; therefore, translation quality and learning quality are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of language processing, and particularly to a translation method, apparatus, device, and medium. Background Art

[0002] With the continuous innovation of science and technology in the field of education, some e-learning products have been launched in the market, such as devices like dictionary pens and electronic dictionaries; these e-learning products integrate a translation function and can use machine translation to achieve instant translation of scanned content, greatly improving the efficiency of users in learning and working.

[0003] However, the content queried by each scan of these e-learning products is very limited, such as a single word, a phrase, a sentence of text, or a passage of text. When translating, it directly performs literal translation based on the limited content collected, and it is difficult to combine the context of the entire text, resulting in a certain difference between the translated content and the context, affecting the translation effect. Summary of the Invention

[0004] The present invention provides a translation method, apparatus, device, and medium to solve the defect that in the prior art, literal translation is directly performed based on the limited content collected during translation, resulting in a certain difference between the translated content and the context, affecting the translation effect, and to achieve the improvement of translation quality.

[0005] The present invention provides a translation method, including: Identifying a target text image to obtain the text content of the text to be translated in the target text image and the book representation feature of the text to be translated, where the book representation feature refers to the feature data representing the book to which the text to be translated belongs; Determining the book type of the book and the context information of the text to be translated in the book according to the book representation feature; Determining the translation display information of the text to be translated according to the text content, the book type, and the context information.

[0006] According to a translation method provided by the present invention, the determining the translation display information of the text to be translated according to the text content, the book type, and the context information includes: Translating the text to be translated according to the text content, the book type, and the context information of the text to be translated to obtain the translation of the text to be translated; Extracting extended learning knowledge of the text to be translated according to the book type to obtain the extended learning information of the text to be translated, where the extended learning information refers to the knowledge information for assisting in learning the text to be translated; Determining the translation display information according to the translation and the extended learning information.

[0007] According to a translation method provided by the present invention, translating the text to be translated according to the text content of the text to be translated, the book type, and the context information to obtain a translation of the text to be translated includes: Determine the translation style information of the text to be translated according to the book type; Determine the context information of the text to be translated according to the context information; Translate the text to be translated according to the translation style information, the context information, and the text content of the text to be translated to obtain the translation.

[0008] According to a translation method provided by the present invention, extracting extended learning knowledge from the text to be translated according to the book type to obtain extended learning information of the text to be translated includes: Determine at least one first extended learning information of the text to be translated according to the book type; When it is determined that the book type is the target book type and the text content of the text to be translated is at the target position, determine the learning content display mode of the text to be translated according to the target book type and the target position; Determine at least one second extended learning information of the text to be translated according to the learning content display mode; Determine the extended learning information according to each of the first extended learning information and each of the second extended learning information.

[0009] According to a translation method provided by the present invention, determining the book type of the book and the context information of the text to be translated in the book according to the book characterization features includes: Identify the book type of the book and the position of the text to be translated in the book according to the book characterization features; Determine the context information in the book according to the position of the text to be translated in the book.

[0010] According to a translation method provided by the present invention, the position of the text to be translated in the book is determined based on the following steps: Determine at least one candidate position in the book according to the book characterization features; Identify the position of the text to be translated in the book from at least one of the candidate positions according to the text content of the text to be translated and the text content in the historical text image associated with the text to be translated; Among them, the historical text image is a text image scanned before the target text image and has an upper-context association with the text to be translated.

[0011] According to a translation method provided by the present invention, the book characterization features are determined based on the following steps: According to the target text image, extract the text format features of the text to be translated, where the text format features include at least one of text shape features, text font size features, text font features, and text color features; According to the target text image, extract the text background features of the text to be translated, where the text background features include image features of the background pattern and / or background color features, and the image features include image color features and / or image structure features; Determine the book characterization features according to the text format features and the text background features.

[0012] The present invention also provides a translation device, including: An identification unit for identifying the target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, where the book characterization features refer to the feature data characterizing the book to which the text to be translated belongs; A determination unit for determining the book type of the book and the context information of the text to be translated in the book according to the book characterization features; A translation unit for determining the translation display information of the text to be translated according to the text content, the book type, and the context information.

[0013] The present invention also provides a learning device, including a scanner, a processor, and a display screen; The scanner is used to scan the image data corresponding to the text to be translated to obtain a target text image; The processor is used to identify the target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, and determine the book type of the book and the context information of the text to be translated in the book according to the book characterization features, and determine the translation display information of the text to be translated according to the text content, the book type, and the context information; The display screen is used to display the translation display information; Among them, the book characterization features refer to the feature data characterizing the book to which the text to be translated belongs.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the program, the translation method described in any one of the above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the translation method described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the translation method described in any one of the above is implemented.

[0017] The translation method, device, equipment, and medium provided by the present invention can, by recognizing a small number of target text images, extract book representation features containing rich feature data characterizing the book to which the text to be translated belongs. Based on the analysis and derivation of the book representation features, the book in which the text to be translated is located, the book type of the book in which the text to be translated is located, and the context information in the book in which the text to be translated is located can be deduced in reverse. Furthermore, the text to be translated can be assisted in translation according to the context information and book type in the book in which the text to be translated is located, which can effectively improve the degree of fit between the translation display information of the text to be translated and the translation style and context of the book in which the text to be translated is located, reduce the difference between the translated content and the context, and thus ensure the translation quality and learning quality. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic flowchart of the translation method provided by the present invention.

[0020] Figure 2 It is a schematic structural diagram of the translation device provided by the present invention.

[0021] Figure 3 It is a schematic structural diagram of the learning device provided by the present invention.

[0022] Figure 4 It is a schematic structural diagram of the electronic device provided by the present invention. Detailed Embodiments

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In order to facilitate students' learning of languages ​​(such as Chinese, English, etc.), some electronic learning products have been launched on the market, such as dictionary pens, electronic dictionaries and other devices. Students can use these electronic learning products to directly look up words. In particular, dictionary pens only require scanning to display the search results for the characters or words they are looking for. Compared with traditional paper dictionaries and manually input electronic dictionaries, these electronic learning products are more convenient in their search methods.

[0025] Existing e-learning products use machine translation to translate scanned content. The accuracy of the translation results depends on the training effect of the translation engine. Although it can greatly improve the efficiency of users in learning and work, the content of the dictionary pen scan query is very limited, such as a word or phrase, a sentence of text, or a paragraph of text (including several sentences of text). The collected text content is very limited. When translating, it is difficult to combine the context of the entire text by directly translating based on the limited content collected, resulting in a certain difference between the translated content and the context, affecting the translation effect, and then affecting the user's learning effect.

[0026] In view of the above problems, this embodiment provides a translation method, which aims to reversely infer the book where the text to be translated is located, the book type of the book where the text to be translated is located, and the context information of the book where the text to be translated is located based on a small amount of text image information, and assist in translating the text to be translated based on the context information and book type of the book where the text to be translated is located, thereby improving the fit between the translation display information of the text to be translated and the translation style and context of the book where the text to be translated is located, improving the translation quality, and then ensuring the learning quality of users.

[0027] Figure 1 : is a flowchart of the translation method provided by the present invention. The method can be applied to a dictionary pen, a scanning pen, or other handheld reading devices with a scanning query function. In addition, the method can also be applied to other types of terminal devices with a query function, such as mobile phones, learning machines, tablet computers, etc., which are not specifically limited in the embodiments of the present invention.

[0028] like Figure 1 As shown, the method includes step 110 , step 120 and step 130 .

[0029] Step 110: Recognize the target text image to obtain the text content of the text to be translated in the target text image and the book representation features of the text to be translated. The book representation features refer to the feature data representing the book to which the text to be translated belongs.

[0030] The target text image here refers to the picture data on the medium where the text to be translated is located. The text to be translated is the text that needs to be translated. The source language of the text to be translated can be Chinese, English, Japanese, etc.; the text to be translated can be a single word, a phrase or a sentence composed of multiple words, or a text formed by combining multiple phrases or sentences. This embodiment does not make specific limitations on this.

[0031] When the execution entity is a dictionary pen, the target text image here can be a spliced image obtained by continuously shooting and scanning the picture data on the medium where the text to be translated is located through the scanner of the dictionary pen, or a text image formed by a user's one-time input through the user interface of the dictionary pen. This embodiment does not make specific limitations on this. The target text image at least includes the text content and book representation features of the text to be translated.

[0032] Optionally, after obtaining the target text image, it can be to recognize the target text image to recognize the text content of the text to be translated in the target text image and the book representation features of the text to be translated. The book representation features here refer to the feature data representing the book to which the text to be translated belongs, including but not limited to multiple items such as text shape features, text size features, text font features, text color features, background color features of the medium where the text is located, and image color features and image structure features of the text background pattern. This embodiment does not make specific limitations on this. The background color feature of the medium where the text is located is also the background color of the page where the text is located; the text background pattern is also the pattern added around or at the bottom of the text.

[0033] Step 120: Determine the book type of the book and the context information of the text to be translated in the book according to the book representation features.

[0034] Optionally, since the publication formats of different types of books are different, therefore, the book, book type to which a part of the content in the book belongs, and the specific position of this part of the content in the book to which it belongs can be deduced in reverse according to the publication format of a part of the content in the book. Furthermore, the context information of this part of the content in the book can be determined based on the specific position.

[0035] Therefore, after obtaining the book representation features of the text to be translated, the book to which the text to be translated belongs, the book type, and the context information of the text to be translated in its belonging book can be identified based on the book representation features of the text to be translated. Here, the book can be Chinese for the first grade, volume 1 (People's Education Edition), English for the fifth grade, volume 2 (Oxford Edition), etc.; the book type can be types such as textbooks, picture books, novels, etc.

[0036] Here, the steps for determining the book to which the text to be translated belongs, the book type, and the context information of the text to be translated in its belonging book can be realized by identifying based on a neural network model or by multi-step logical inference. This implementation does not make specific limitations on this. For example, when realizing by identifying based on a neural network model, the book representation features can be input into a pre-trained neural network model with multi-task text recognition to identify and realize the determination of the book to which the text to be translated belongs, the book type, and the context information of the text to be translated in its belonging book. The multi-task text recognition here at least includes the recognition tasks of the book to which the text to be translated belongs, the book type, and the context information of the text to be translated in its belonging book. Another example, when realizing by multi-step logical inference, it can be to first perform correlation matching by combining the multi-dimensional representation features in the book representation features with the representation features of different books, so as to determine the book that matches the representation features of the book to which the text to be translated belongs as the book to which the text to be translated belongs, and perform category division on the content of the book to obtain the corresponding book type, and combine the text content of the text to be translated and the book representation features, or combine the text content of the text to be translated, the book representation features, and the text image with which the text to be translated has context association, etc., to determine the position of the text to be translated in its belonging book, so as to extract context information from the book to which the text to be translated belongs according to this position.

[0037] Step 130, determine the translation display information of the text to be translated according to the text content, the book type, and the context information.

[0038] Optionally, after obtaining the text content, book type, and context information of the text to be translated, the text to be translated can be translated by combining the text content, book type, and context information of the text to be translated to obtain the translation of the text to be translated, and the translation can be directly used as the translation display information for display. Here, the translation is the text obtained by translating the text to be translated. The language to which the translation belongs, that is, the translated language, is the target language of the source language. The target language is a language different from the source language. For example, if the source language and the target language are English and Chinese respectively, English-to-Chinese translation can be achieved.

[0039] Here, when obtaining the translation of the text to be translated, it can be to input the text content, book type, and context information of the text to be translated into a natural language model (such as a large language model) for translation, so as to obtain a translation with a relatively high degree of fit with the translation style and context of the book where the text to be translated is located; or, it is to match the text content, book type, and context information of the text to be translated with the text content, book type, and context information of the translated text fragments stored in the translation memory, and extract the translation of the text to be translated according to the matching result.

[0040] In addition, in order to assist users in effectively learning translation knowledge while also enabling synchronous learning and understanding of the text to be translated to further improve the learning quality, it can also be to mine the extended learning information of the text to be translated in combination with the book type of the text to be translated when obtaining the translation, and display the translation and the extended learning information together as the translation display information. Thus, while providing a more accurate translation to the user, rich extended learning knowledge is provided, thereby enhancing the learning quality.

[0041] The method provided in this embodiment can extract the book representation features containing rich feature data representing the book to which the text to be translated belongs by recognizing a small number of target text images. Based on the analysis and derivation of the book representation features, the book where the text to be translated is located, the book type of the book where the text to be translated is located, and the context information in the book where the text to be translated is located can be deduced in reverse. Furthermore, the text to be translated can be assisted in translation according to the context information and book type in the book where the text to be translated is located, which can effectively improve the degree of fit between the translation display information of the text to be translated and the translation style and context of the book where the text to be translated is located, reduce the difference between the translated content and the context, and thus ensure the translation quality and learning quality.

[0042] In some embodiments, step 130 specifically includes: Translate the text to be translated according to the text content, book type, and context information of the text to be translated to obtain the translation of the text to be translated; Extract extended learning knowledge of the text to be translated according to the book type to obtain the extended learning information of the text to be translated, where the extended learning information refers to the knowledge information for assisting in learning the text to be translated; Determine the translation display information according to the translation and the extended learning information.

[0043] It should be noted that when the content scanned and queried by a dictionary pen in the related art is a sentence of text or a paragraph of text, the query result provided by the dictionary pen is relatively single, only the translation. This simple way of providing translation content limits the learning assistance provided by the dictionary pen to users, lacks diversified learning assistance, and affects the learning quality.

[0044] In view of this problem, when performing the translation task in this embodiment, in addition to inferring the book where the text to be translated is located, the book type of the book where the text to be translated is located, and the context information in the book where the text to be translated is located from a small amount of picture information, so as to assist in translating the text to be translated according to the context information and book type in the book where the text to be translated is located and improve the translation quality, it can also provide extended learning information that conforms to the book type and the context semantics of the position where the text to be translated is located for the text to be translated according to the book type, so as to assist the user in understanding the query content and improve the learning quality.

[0045] Optionally, when obtaining the translation display information, it can be to translate the text to be translated by combining the text content, book type, and context information of the text to be translated to obtain the translation of the text to be translated. At the same time, it can also extract extended learning knowledge for the text to be translated based on the book type to obtain the extended learning information of the text to be translated; thus, by combining the translation obtained by translation and the extended learning information obtained by knowledge extraction, the translation display information is determined, so that when performing the translation task, the translation and the extended learning information can be synchronously displayed to the user, thereby providing the user with accurate and rich query results, enabling the user to more deeply understand the query content, optimizing the user experience, and improving the learning quality.

[0046] The extended learning information here refers to the knowledge information that assists the user in learning the text to be translated, including but not limited to one or more of sentence explanation information, pronunciation recitation information, semantic expression information, synonymous replacement information, dialogue context continuity information, emotion or mood analysis information, pronunciation information, translation explanation information, and audio playback items. This embodiment does not make specific limitations on this.

[0047] When determining the extended learning information, it can be directly mapped and matched according to the book type to obtain the extended learning information, or on the basis of mapping and matching to obtain the first extended learning information according to the book type, and then introduce a specific learning content display mode determined according to the book type and the position of the text content of the text to be translated for mapping and matching to obtain the second extended learning information, and form the extended learning information by combining the first extended learning information and the second extended learning information, etc. This embodiment does not make specific limitations on this.

[0048] The method provided in this embodiment performs translation by considering the text content, book type, and context information, and extracts extended learning information, so that when displaying the query results, it can not only display the translation to the user, but also display the extended learning information related to the query content to the user, providing the user with accurate and rich query results, assisting the user in better understanding the text to be translated, the translation, and the context content, improving the understandability of the learning content by the user, optimizing the user experience, and improving the learning quality.

[0049] In some embodiments, translating the text to be translated according to the text content of the text to be translated, the book type, and the context information to obtain a translation of the text to be translated includes: Determining translation style information for the text to be translated according to the book type; Determining context information for the text to be translated according to the context information; Translating the text to be translated according to the translation style information, the context information, and the text content of the text to be translated to obtain the translation.

[0050] Optionally, when performing text translation, since different book types, such as novels, academic works, news reports, children's books, etc., often require different language styles and expression forms, the translation style information for the text to be translated can be determined first according to the book type. For example, specifically, the translation style information for the text to be translated can be obtained by performing mapping and matching according to the mapping relationship between the book type and the translation style.

[0051] In addition, in order to further understand the context where the text to be translated is located, the context of the text to be translated can also be analyzed according to the context information to obtain the context information of the text to be translated.

[0052] After obtaining the translation style information and context information of the text to be translated, the text to be translated can be translated by combining the text content, translation style information, and context information of the text to be translated to obtain a translation of the text to be translated.

[0053] It should be noted that when obtaining the translation of the text to be translated, the initial translation text of the text to be translated can be extracted from the translation text corresponding to the context information according to the text content of the text to be translated, and then the initial translation text can be polished and adjusted according to the translation style information to obtain the translation of the text to be translated; or the translation style information, context information, and text content of the text to be translated can be input into a translation model (such as a large language model), and the translation model performs the translation, thereby obtaining a translation that conforms to the translation style of the text to be translated and is adapted to the context of the text to be translated, so as to improve the translation quality.

[0054] The method provided in this embodiment determines the translation style in combination with the book type, analyzes the context using the context information, and performs text translation by combining the text content, achieving a translation effect with a matching translation style and high context fit with the book type, and significantly improving the translation quality and accuracy.

[0055] In some embodiments, the extracting extended learning information for the text to be translated according to the book type includes: Determine at least one first extended learning information of the text to be translated according to the type of the book. When it is determined that the type of the book is the target book type and the text content of the text to be translated is at the target position, determine the learning content display mode of the text to be translated according to the target book type and the target position. Determine at least one second extended learning information of the text to be translated according to the learning content display mode. Determine the extended learning information according to each of the first extended learning information and each of the second extended learning information.

[0056] It should be noted that since the emphasis of the knowledge information that users need to learn is different for the same text in different types of books, when performing text translation, in order to improve the learning quality of users, in addition to providing users with accurate translations, extended learning information that conforms to the book content can also be provided to assist users in better understanding the text to be translated, the translation, and the context, improving the understandability of the learning content by users, optimizing the user experience, and enhancing the learning quality.

[0057] Optionally, since the extended learning dimensions corresponding to different book types are different, and the association mapping relationships between different book types and different extended learning dimensions are established in advance.

[0058] Table 1 Association mapping relationship table between different book types and different extended learning dimensions

[0059] For example, the extended learning dimensions corresponding to textbooks include, but are not limited to, pronunciation recitation, sentence explanation, semantic expression, synonym replacement, and analysis of the continuity of the dialogue context; the extended learning dimensions corresponding to picture books include, but are not limited to, pronunciation recitation, and analysis of the emotion or mood of sentences; the extended learning dimensions corresponding to novels include, but are not limited to, analysis of the correctness of the semantic translation of the context. This embodiment does not make specific limitations, and the specific mapping relationship can be seen in Figure 1.

[0060] Therefore, when obtaining the extended learning information of the text to be translated, the extended learning dimension of the text to be translated can be searched by association according to the type of the book to which the text to be translated belongs and the association mapping relationship between different book types and different extended learning dimensions, and corresponding annotations can be added to the text to be translated through this extended learning dimension, so as to obtain at least one first extended learning information.

[0061] In addition, the book type of the book to which the text to be translated belongs can be matched with the target book type. When it is determined that the book type of the book to which the text to be translated belongs is the target book type, since the learning content display modes of different parts within the target book type are different. Therefore, when it is determined that the book type of the book to which the text to be translated belongs is the target book type, the position of the text content of the text to be translated in the book can also be matched with the target position. When it is further determined that the text content of the text to be translated is located at the target position, the learning content display mode of the text to be translated is obtained through associated mapping based on the target book type and the target position; and based on the associated mapping relationship between the learning content display mode and the extended learning dimension, the extended learning dimension of the text to be translated is further mapped and obtained. Then, corresponding annotations are added to the text to be translated through this extended learning dimension, and at least one second extended learning information can be obtained.

[0062] The target book type and the target position here can be set according to actual requirements. For example, the target book type can be a textbook type, etc., and the target position can be the main text part or the singing section, etc. This embodiment does not make specific limitations on this.

[0063] The following uses a specific example to describe the steps for obtaining the second extended learning information.

[0064] For example, when it is determined that the book type of the book to which the text to be translated belongs is a textbook type, since the learning content display modes of different parts within the textbook are different. For example, the display dimensions corresponding to the learning content display mode of "Good morning" in the main text part include pronunciation, translation explanation, and semantic explanation, while the display dimensions corresponding to the learning content display mode in the singing section include a music playback option. Therefore, when it is further determined that the text content of the text to be translated is located in the main text part, pronunciation information, translation explanation information, and semantic explanation information are marked for the text to be translated to obtain the second extended learning information; when it is further determined that the text content of the text to be translated is located in the singing section, a music playback item is marked for the text to be translated to obtain the second extended learning information.

[0065] After obtaining the first extended learning information and the second extended learning information, it is possible to integrate each first extended learning information and each second extended learning information to obtain the extended learning information of the text to be translated, so as to provide accurate and rich query results for users, better improve the user's understandability of the learning content, optimize the user experience, and enhance the learning quality.

[0066] In some embodiments, step 120 specifically includes: According to the book characterization features, identify the book type of the book and the position of the text to be translated in the book; Determine the context information in the book according to the position of the text to be translated in the book.

[0067] Optionally, after obtaining the book representation features of the text to be translated, it may be based on the book representation features to identify the book to which the text to be translated belongs, and classify the content of the book to obtain the corresponding book type. Here, identifying the book to which the text to be translated belongs can be implemented based on a language model, such as a large language model, or can be implemented by matching the book representation features of the text to be translated with the representation features of different types of books. This embodiment does not make specific limitations on this.

[0068] In addition, the book representation features and the text content of the text to be translated can be combined to further locate the position of the text to be translated in the book to which it belongs, so as to obtain the position of the text to be translated in the book to which it belongs; or on the basis of combining the book representation features and the text content of the text to be translated, text images and the like that have a context association with the text to be translated can be further introduced to locate the position of the text to be translated in the book to which it belongs, so as to obtain the position of the text to be translated in the book to which it belongs.

[0069] After obtaining the position of the text to be translated in the book to which it belongs, the front and back text content of the text to be translated can be extracted from the book to which the text to be translated belongs according to this position, so as to obtain the context information of the text to be translated.

[0070] The method provided in this embodiment accurately identifies the book type, locates the position of the text to be translated in the book, and extracts the context information of the text to be translated through the book representation features and the text content of the text to be translated, and accordingly can effectively improve the translation quality.

[0071] In some embodiments, the position of the text to be translated in the book is determined based on the following steps: Determine at least one candidate position in the book according to the book representation features; Identify the position of the text to be translated in the book from at least one of the candidate positions according to the text content of the text to be translated and the text content in the historical text image associated with the text to be translated; Wherein, the historical text image is a text image that is scanned before the target text image and has an upstream association with the text to be translated.

[0072] Optionally, when determining the position of the text to be translated in the book to which it belongs, it is possible to first perform a coarse-grained matching of the book representation features with the book representation features of the text content at each position in the book to which the text to be translated belongs, so as to obtain at least one candidate position with a relatively high matching degree with the book representation features. Then, in order to more accurately locate the position of the text to be translated, the text content of the text to be translated and the text content in the historical text image associated with the text to be translated can be further combined, and through text matching and context analysis, the position of the text to be translated can be refined to accurately obtain the position of the text to be translated in the book to which it belongs.

[0073] The refined positioning here can be achieved by inputting the text content of the text to be translated, the text content in the historical text image associated with the text to be translated, and the text content corresponding to the candidate position into a language model for recognition and positioning, or can be achieved by performing text matching on the text content of the text to be translated, the text content in the historical text image associated with the text to be translated, and the text content corresponding to the candidate position. This embodiment does not make specific limitations in this regard.

[0074] Here, the historical text image associated with the text to be translated refers to the text image that needs to be text-translated and has an upstream association with the text to be translated, which is scanned before obtaining the target text image, that is, before performing step 110. The text content it contains belongs to the same book as the text content of the text to be translated; this historical text image can be obtained by a single scan or by continuous multiple scans. This embodiment does not make specific limitations in this regard.

[0075] The method provided in this embodiment, by combining the coarse-grained matching of book representation features, the refined text matching of the text to be translated and historical text content, and context analysis, can efficiently and accurately determine the specific position of the text to be translated in the book.

[0076] In some embodiments, the book representation features described in step 110 are determined based on the following steps: According to the target text image, extract the text format features of the text to be translated, where the text format features include at least one of text shape features, text size features, text font features, and text color features; According to the target text image, extract the text background features of the text to be translated, where the text background features include the image features of the background pattern and / or the background color features, and the image features include the image color features and / or the image structure features; According to the text format features and the text background features, determine the book representation features.

[0077] Optionally, when extracting book representation features, text format feature recognition may be performed on the target text image to extract at least one of a text shape feature, a text size feature, a text font feature, and a text color feature of the text to be translated from the target text image, thereby obtaining the text format feature of the text to be translated; In addition, the target text image may be subjected to text background feature recognition to extract image features of the background pattern and / or background color features of the background where the text to be translated is located from the target text image, thereby obtaining text format features of the text to be translated.

[0078] Then, the text format features and text background features of the text to be translated are integrated to obtain the book representation features of the text to be translated, thereby obtaining rich feature data of the book to which it belongs, thereby ensuring the comprehensiveness, accuracy and reliability of the query results.

[0079] The translation device provided by the present invention is described below. The translation device described below and the translation method described above can be referred to each other.

[0080] Figure 2 is a schematic diagram of the structure of the translation device provided by the present invention; Figure 2 As shown, the device comprises: The recognition unit 210 is used to recognize the target text image, and obtain the text content of the text to be translated in the target text image and the book characterization feature of the text to be translated, wherein the book characterization feature refers to the feature data characterizing the book to which the text to be translated belongs; The determination unit 220 is used to determine the book type of the book and the context information of the text to be translated in the book according to the book characterization feature; The translation unit 230 is used to determine the translation display information of the text to be translated according to the text content, the book type and the context information.

[0081] The device provided in this embodiment can extract book representation features containing rich feature data representing the book to which the to-be-translated text belongs by recognizing a small amount of target text images, and can reversely deduce the book to which the to-be-translated text belongs, as well as the book type of the book to which the to-be-translated text belongs and the context information in the book to which the to-be-translated text belongs, based on the analysis and deduction of the book representation features. Then, the to-be-translated text can be assisted in translation based on the context information and book type in the book to which the to-be-translated text belongs, which can effectively improve the degree of fit between the translation display information of the to-be-translated text and the translation style and context of the book to which the to-be-translated text belongs, reduce the difference between the translation content and the context, and thereby ensure the translation quality and learning quality.

[0082] In some embodiments, the translation unit is specifically used to: Translate the text to be translated according to the text content of the text to be translated, the type of the book, and the context information, to obtain the translation of the text to be translated; Extract extended learning knowledge from the text to be translated according to the type of the book, to obtain the extended learning information of the text to be translated, where the extended learning information refers to the knowledge information for assisting in learning the text to be translated; Determine the translation display information according to the translation and the extended learning information.

[0083] In some embodiments, the translation unit is further configured to: Determine the translation style information of the text to be translated according to the type of the book; Determine the context information of the text to be translated according to the context information; Translate the text to be translated according to the translation style information, the context information, and the text content of the text to be translated, to obtain the translation.

[0084] In some embodiments, the translation unit is further configured to: Determine at least one first extended learning information of the text to be translated according to the type of the book; In the case where the type of the book is determined to be the target book type and the text content of the text to be translated is located at the target position, determine the learning content display mode of the text to be translated according to the target book type and the target position; Determine at least one second extended learning information of the text to be translated according to the learning content display mode; Determine the extended learning information according to each of the first extended learning information and each of the second extended learning information.

[0085] In some embodiments, the determining unit is specifically configured to: Identify the type of the book and the position of the text to be translated in the book according to the book characterization features; Determine the context information in the book according to the position of the text to be translated in the book.

[0086] In some embodiments, the determining unit is further configured to: Determine at least one candidate position in the book according to the book characterization features; Identify the position of the text to be translated in the book from at least one of the candidate positions according to the text content of the text to be translated and the text content in the historical text image associated with the text to be translated; Among them, the historical text image is a text image scanned before the target text image and has an upper-context association with the text to be translated.

[0087] In some embodiments, the recognition unit is specifically configured to: Extract the text format features of the text to be translated according to the target text image, where the text format features include at least one of text shape features, text font size features, text font features, and text color features; Extract the text background features of the text to be translated according to the target text image, where the text background features include image features of the background pattern and / or background color features, and the image features include image color features and / or image structure features; Determine the book characterization features according to the text format features and the text background features.

[0088] In some embodiments, the present application further provides a learning device. As Figure 3 is a schematic structural diagram of the learning device provided by the present invention; as Figure 3 shown, the learning device 300 includes a scanner 310, a processor 320, and a display screen 330; The scanner 310 is configured to scan the image data corresponding to the text to be translated to obtain a target text image; The processor 320 is configured to recognize the target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, and determine the book type of the book and the context information of the text to be translated in the book according to the book characterization features, and determine the translation display information of the text to be translated according to the text content, the book type, and the context information; The display screen 330 is configured to display the translation display information; Among them, the book characterization features refer to the feature data characterizing the book to which the text to be translated belongs.

[0089] The learning device 300 here may be a learning product such as a dictionary pen or an electronic dictionary, and this embodiment does not specifically limit this.

[0090] Optionally, a scanner 310 is provided on the learning device 300, and the scanner 310 is configured to scan and capture the picture data on the medium where the text to be translated is located to obtain a target text image.

[0091] The learning device 300 is also provided with a processor 320; after obtaining the target text image, the processor 320 can identify the target text image to obtain the text content of the text to be translated in the target text image and the book representation features of the text to be translated, and determine the book type of the book and the context information of the text to be translated in the book according to the book representation features, and determine the translation display information of the text to be translated according to the text content, book type and context information, thereby realizing the translation task processing of the text to be translated.

[0092] In addition, a display screen 330 is also provided on the learning device 300. The display screen 330 can display the translation display information processed by the processor 320, so that students can better understand the content of the text to be translated and learn the translation, improving the learning quality.

[0093] The learning device provided in this embodiment can identify a small number of target text images through the processor, extract the book representation features containing rich characteristic data representing the book to which the text to be translated belongs, and on the basis of analyzing and deducing the book representation features, can inversely deduce the book where the text to be translated is located, the book type of the book where the text to be translated is located, and the context information in the book where the text to be translated is located. Furthermore, the text to be translated can be assisted in translation according to the context information and book type in the book where the text to be translated is located, which can effectively improve the matching degree between the translation display information of the text to be translated and the translation style and context of the book where the text to be translated is located, reduce the difference between the translated content and the context, and thus ensure the translation quality and learning quality.

[0094] The device and learning device provided by the present invention are used to execute the above method embodiments. For the specific process and detailed content, please refer to the above embodiments and will not be elaborated here.

[0095] Figure 4 An example of the physical structure diagram of an electronic device is as Figure 4As shown in the figure, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communications interface 420, and the memory 430 complete their mutual communication through the communication bus 440. The processor 410 may call the logical instructions in the memory 430 to execute a translation method, which includes: recognizing a target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, where the book characterization features refer to the feature data characterizing the book to which the text to be translated belongs; determining the book type of the book and the context information of the text to be translated in the book according to the book characterization features; and determining the translation display information of the text to be translated according to the text content, the book type, and the context information.

[0096] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0097] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the translation method provided by the above-mentioned various methods. The method includes: recognizing a target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, where the book characterization features refer to the feature data characterizing the book to which the text to be translated belongs; determining the book type of the book and the context information of the text to be translated in the book according to the book characterization features; and determining the translation display information of the text to be translated according to the text content, the book type, and the context information.

[0098] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the translation method provided by the above-mentioned various methods. The method includes: identifying a target text image to obtain the text content of the text to be translated in the target text image and the book representation feature of the text to be translated, where the book representation feature refers to the feature data representing the book to which the text to be translated belongs; determining the book type of the book and the context information of the text to be translated in the book according to the book representation feature; and determining the translation display information of the text to be translated according to the text content, the book type and the context information.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A translation method, characterized in that, Including: Identifying a target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, where the book characterization features refer to the feature data characterizing the book to which the text to be translated belongs; Determining the book type of the book and the context information of the text to be translated in the book according to the book characterization features; Determining the translation display information of the text to be translated according to the text content, the book type and the context information.

2. The translation method according to claim 1, characterized in that, The determining the translation display information of the text to be translated according to the text content, the book type and the context information includes: Translating the text to be translated according to the text content, the book type and the context information of the text to be translated to obtain the translation of the text to be translated; Performing extended learning knowledge extraction on the text to be translated according to the book type to obtain the extended learning information of the text to be translated, where the extended learning information refers to the knowledge information for assisting in learning the text to be translated; Determining the translation display information according to the translation and the extended learning information.

3. The translation method according to claim 2, characterized in that, The translating the text to be translated according to the text content, the book type and the context information of the text to be translated to obtain the translation of the text to be translated includes: Determining the translation style information of the text to be translated according to the book type; Determining the context information of the text to be translated according to the context information; Translating the text to be translated according to the translation style information, the context information and the text content of the text to be translated to obtain the translation.

4. The translation method according to claim 2, characterized in that The performing extended learning knowledge extraction on the text to be translated according to the book type to obtain the extended learning information of the text to be translated includes: Determining at least one first extended learning information of the text to be translated according to the book type; In the case where it is determined that the book type is the target book type and the text content of the text to be translated is at the target position, determining the learning content display mode of the text to be translated according to the target book type and the target position; Determining at least one second extended learning information of the text to be translated according to the learning content display mode; Determining the extended learning information according to each of the first extended learning information and each of the second extended learning information.

5. The translation method according to any one of claims 1-4, characterized in that, The determining the book type of the book and the context information of the text to be translated in the book according to the book characterization features includes: Identifying the book type of the book and the position of the text to be translated in the book according to the book characterization features; Determining the context information in the book according to the position of the text to be translated in the book.

6. The translation method according to claim 5, characterized in that, The position of the text to be translated in the book is determined based on the following steps: Determining at least one candidate position in the book according to the book characterization features; Identify the position of the text to be translated in the book from among at least one of the candidate positions, based on the text content of the text to be translated and the text content in the historical text image associated with the text to be translated. The historical text image is a text image scanned before the target text image and having an upper-context association with the text to be translated.

7. The translation method according to any one of claims 1 to 4, characterized in that The book characterization features are determined based on the following steps: Extract the text format features of the text to be translated from the target text image, where the text format features include at least one of text shape features, text font size features, text font features, and text color features. Extract the text background features of the text to be translated from the target text image, where the text background features include the image features of the background pattern and / or background color features, and the image features include image color features and / or image structure features. Determine the book characterization features based on the text format features and the text background features.

8. A translation device, characterized in that, Comprising: An identification unit configured to identify a target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, where the book characterization features refer to the feature data characterizing the book to which the text to be translated belongs. A determination unit configured to determine the book type of the book and the context information of the text to be translated in the book based on the book characterization features. A translation unit configured to determine the translation display information of the text to be translated based on the text content, the book type, and the context information.

9. A learning device, characterized in that, Comprising a scanner, a processor, and a display screen; The scanner is configured to scan the image data corresponding to the text to be translated to obtain a target text image. The processor is configured to identify the target text image to obtain the text content of the text to be translated in the target text image and the book characterization features of the text to be translated, and determine the book type of the book and the context information of the text to be translated in the book based on the book characterization features, and determine the translation display information of the text to be translated based on the text content, the book type, and the context information. The display screen is configured to display the translation display information. The book characterization features refer to the feature data characterizing the book to which the text to be translated belongs.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the translation method according to any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the translation method according to any one of claims 1 to 7.

Images