Content translation method, system and wearable translation device

Abstract

The present application provides a content translation method, system and wearable translation device, including obtaining first content, the first content including at least a first language; converting the first content into second content by a first conversion, the second content including a second language, and the second language is different from the first language; generating third content according to the second content, the third content including at least: a reply text in reply to the second content, the reply text consisting of the second language; a translation text generated based on the reply text, the translation text obtained by the first conversion, the translation text consisting of the first language; a phonetic text based on the translation text, the phonetic text obtained by a second conversion different from the first conversion, the phonetic text including the second language and the pronunciation being the same as or similar to the first language; presenting the translation text and the phonetic text. The above scheme can realize effective conversion between different languages, and users can improve communication efficiency through their own language expression, further shortening the emotional distance between users.

Description

Technical Field

[0001] The present application belongs to the field of translation technology, and specifically relates to a content translation method, system and wearable translation device. Background Art

[0002] Translation refers to the process of converting text or spoken expressions in one language into another language, or the result of such a process. The purpose of translation is to enable communication and understanding between different languages. Translation can cover written texts, oral expressions, professional terms, and culturally specific elements. The translation process requires the translator to have good language skills, cultural background knowledge, and understanding to ensure that the information conveyed is accurate, fluent, and in line with the language habits and cultural background of the target language.

[0003] With the popularization of computers and mobile terminals, there are devices and applications that can perform real-time voice translation. For example, existing handheld translation machines or smartphone applications generally translate the wearer's language into the target language to be conveyed, and then let the communication partner watch the translation on the device, or directly play the translated language through the speaker. Conversely, if the wearer needs to understand the communication partner's language, he or she will continue to receive the language expressed by the communication partner and convert it into a language that the wearer can understand, and let the wearer watch the translated content or directly. The above translation and communication process requires both parties to watch the translated content repeatedly, and the communication efficiency is low. Summary of the invention

[0004] In order to overcome the defects of the above-mentioned prior art, in a first aspect, the present application proposes a content translation method, comprising: obtaining a first content, the first content including at least a first language; converting the first content into a second content by a first conversion, the second content including a second language, and the second language is different from the first language; generating a third content according to the second content, wherein the third content at least includes: a reply text in reply to the second content, the reply text consisting of the second language; a translation text generated based on the reply text, the translation text obtained by the first conversion, the translation text consisting of the first language; a phonetic text based on the translation text, the phonetic text obtained by a second conversion different from the first conversion, the phonetic text including the second language and the pronunciation being the same as or similar to the first language; and presenting the translation text and the phonetic text.

[0005] In a second aspect, the present application provides a content translation system, comprising: one or more processors,

[0006] A memory, wherein the memory stores program instructions, and the program instructions can be executed by the one or more processors, and the operations include: obtaining a first content, the first content including at least a first language; converting the first content into a second content by a first conversion, the second content including a second language, and the second language is different from the first language; generating a third content according to the second content, wherein the third content includes at least: a reply text in reply to the second content, the reply text consisting of the second language; a translation text generated based on the reply text, the translation text obtained by the first conversion, and the translation text consisting of the first language; a phonetic text based on the translation text, the phonetic text obtained by a second conversion different from the first conversion, the phonetic text including the second language and the pronunciation being the same as or similar to the first language; and presenting the translation text and the phonetic text on a display module.

[0007] In a third aspect, the present application provides a wearable translation device, comprising a housing;

[0008] at least one processor located in the housing;

[0009] A memory storing instructions, the processor being configured to execute the instructions stored in the memory, wherein the instructions, when executed by the processor, configure the processor to: obtain first content, the first content including at least a first language; convert the first content into second content by a first conversion, the second content including a second language, the second language being different from the first language; generate third content according to the second content, wherein the third content includes at least: a reply text in reply to the second content, the reply text being in the second language; a translation text generated based on the reply text, the translation text being obtained by the first conversion, the translation text being in the first language; a phonetic text based on the translation text, the phonetic text being obtained by a second conversion different from the first conversion, the phonetic text including the second language and having the same or similar pronunciation as the first language; and present the translation text and the phonetic text on a display module.

[0010] The solution proposed in this application can effectively convert and communicate between the user's two languages, is suitable for cross-language communication scenarios, can help users understand and express information more easily, and provide phonetic text corresponding to the translated content in the original language to help users more easily master the correct pronunciation and intonation when expressing, thereby increasing user interactivity and participation.

[0011] These and other features and advantages thereof will be more clearly understood from the following detailed description in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A system block diagram of the operating environment for content translation provided in an embodiment of the present application;

[0014] Figure 2 A schematic diagram of the structure of an embodiment of a wearable translation device provided in an embodiment of the present application;

[0015] Figure 3 A schematic diagram of the display module structure of a wearable translation device provided in an embodiment of the present application;

[0016] Figure 4 A schematic diagram of the architecture of a wearable translation device provided in an embodiment of the present application;

[0017] Figure 5 A schematic structural diagram of another embodiment of a wearable translation device provided in an embodiment of the present application;

[0018] Figure 6 A schematic diagram of a business process of a content translation method provided in an embodiment of the present application;

[0019] Figure 7 Another business process diagram of the content translation method provided in the embodiment of the present application;

[0020] Figure 8 A flowchart of a content translation method provided in an embodiment of the present application;

[0021] Fig. 9 A schematic diagram of presenting content in a first display area using a content translation method provided in an embodiment of the present application;

[0022] Fig.10 A schematic diagram of presenting content in the second display area using the content translation method provided in an embodiment of the present application;

[0023] Fig.11 A schematic diagram of content presentation in an application scenario of a content translation method provided in an embodiment of the present application;

[0024] Fig.12 A schematic diagram of content presentation in another application scenario of the content translation method provided in an embodiment of the present application;

[0025] Fig.13 A schematic diagram of the structure of a content translation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0028] The terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0029] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "one embodiment", "some embodiments", "another embodiment", "some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0030] Figure 1The content translation operating environment architecture block diagram provided by the embodiment of the present application, according to some embodiments, the system has an application ecosystem for processing content data input. Among them, the operating environment 100 includes a wearable device or wearable translation device 101, a network 102, an external terminal device 103, an external cloud computing platform or server 104, and a local server 105. The wearable device 101 can first be connected to the terminal device 103 through the network 102, and the terminal device 103 is connected to the external cloud computing platform or server 104 or the local server 105 through the network 102, thereby realizing that the wearable translation device 101 can be connected to the external cloud computing platform or server 104. Among them, the wearable device 101, the (external) terminal device 103, the external cloud computing platform or server 104, and the local server 105 can each be used to store and process content data received or uploaded by the wearable device 101; in some embodiments, the wearable device 101 can be directly connected to the external cloud computing platform or server 104 or the local server 105 through the network 102. The wearable device 101 may be a portable, movable, lightweight device, such as a near-eye display device such as smart glasses or augmented reality devices, which may include a display module 110, and the terminal device 103 may be a smart phone, a laptop, a tablet or a desktop computer, etc. In some embodiments, the wearable devices 101 may be communicatively connected to each other, and in some embodiments, the wearable translation device 101 may be communicatively connected to wearable interactive devices such as smart rings or bracelets or handles, and interactive operations on the wearable translation device 101 may be implemented through such interactive devices, such as page turning, page sliding, selection or confirmation, and other interactive operations.

[0031] In some embodiments, in combination with Figure 2-Figure 3 The wearable translation device 101 may include a near-eye display device, which includes a frame 120 and a temple 130 connected to the frame 120, wherein the display module 110 may be arranged on the frame 120, for example, at the upper frame edge. In other embodiments, the display module 110 may also be installed on the lens 140 or the frame 120 in a detachable or removable manner. The wearable device 101 may include an environmental side 150 and a near-eye side 160, wherein the digital content of the display module 110 faces the near-eye side 160, thereby improving the concealment of the translation content. In some embodiments, the temple 130 may be provided with components such as a battery and a circuit board (not shown in the figure), such as Figure 3As shown, the display module 110 may include a front cover 1103 and a back cover 1102, the micro display 1101 is located in the accommodation cavity formed by the front cover 1103 and the back cover 1102, wherein the micro display 1101 can also be electrically connected to the battery or circuit board and other electrical components of the temple 130 through the flexible circuit board 1110, the optical module 1108 is located on the light emitting side of the micro display 1101, the outer periphery of the optical module 1108 can be provided with an external thread 1109, the front cover 1103 is provided with a first through hole 1104, the inner peripheral wall of the first through hole 1104 is provided with an internal thread 1105, and the flexible The sleeve 1106 is provided with a second through hole 1107, and the flexible sleeve 1106 is located between the optical module 1108 and the front cover 1103, wherein the external thread 1109 and the internal thread 1105 are threadedly connected to each other to squeeze and deform the flexible sleeve 1106, thereby effectively fixing the optical module 1108 to the light-emitting side of the microdisplay 1101, wherein the optical module 1108 can adopt the optical solution of CN202322496484.0, wherein the light of the microdisplay 1101 can pass through the optical module 1108 and be projected toward the near-eye side 160 to the user's eyes. Among them, the micro display 1101 may include Micro-LED (Micro Light-Emitting Diode), uLED (micro light-emitting diode), Micro-oled (Micro Organic Light-Emitting Diode), LCoS (Liquid Crystal On Silicon), LCD (Liquid Crystal Display), DMD (Digital Micromirror Device) / DLP (Digital Light Processing) or LBS (Laser Beam Scanning), etc., or any combination of the above products. The optical module 1108 can be cylindrical, and its projection area along the optical axis can be an ellipse, a circle or a polygon. Therefore, the outline of the translation content presentation area on the near-eye display device can be a circle, an ellipse or a polygon, etc.

[0032] In other embodiments, Figure 5 As shown, an optical solution such as a diffraction light waveguide can be used to directly present digital content on the lens 140. In this case, the optical machine (or micro display) can be located at the temple 130 or near the temple 130. Through this method, the translated text and phonetic text and other related presentations can be directly performed in front of the lens 140 or the main field of view of the human eye. In other embodiments, head-mounted near-eye display devices such as virtual reality display devices or mixed reality display devices are also feasible.

[0033] Combined with reference Figure 4 The architecture example diagram of the wearable device or translation device 101 in the embodiment of the present invention is as follows. In some embodiments, the wearable device 101 may include one or more processors 1011, one or more memories 1012, a microdisplay 1101, a battery 1013, a communication unit 1015, a first microphone 1016, a second microphone 1017, a first speaker 1018, a second speaker 1019, a position sensor 1020, an optical sensor 1021, a biometric sensor 1022, a user interaction input 1023, an electrical interface 1014, an inertia sensor 1024 and a camera system 1025. The above components may be integrated into one or more integrated circuits, which may be electrically coupled to each other by multiple circuit boards in terms of physical structure.

[0034] In some embodiments, the memory 1012 may include a high-speed random access memory or a non-volatile memory, such as one or more disk storage devices, one or more optical storage devices, or flash memory. It stores an instruction set 10120 for processing basic system services and performing hardware-related tasks. The instruction set 10120 may be an instruction set for facilitating the processing of related sensors or interfaces. The memory 1012 may also store an operating system, such as Darwin, RTXC (real-time multiprocessor execution system), LINUX, UNIX (Eunice), Android, iOS, WINDOWS, or other embedded operating systems. The memory 1012 stores one or more programs configured to be executed by the one or more processors 1011, and the one or more programs include an instruction set for operation.

[0035] In some embodiments, the communication unit 1015 may include wireless communication, and may include a radio frequency receiver and transmitter and / or an optical (e.g., infrared) receiver and transmitter, which is designed to communicate via a mobile communication (GSM) network, a GPRS network, an Enhanced Data GSM Environment (EDGE) network, an IEEE802.xx communication network (e.g., WiFi, WiMax, ZigBee TM), 3G, 4G, 4G LTE, 5G, code division multiple access (CDMA) network, near field communication (NFC), WiFi direct, infrared and Bluetooth network, etc., or a global system in combination. The wireless communication unit may include a hosting protocol so that the device can be configured as a base station for other wireless devices. For another example, the communication subsystem may allow the device to synchronize with the host device using one or more protocols or communication technologies, such as, for example, TCP / IP (Transmission Control Protocol / Internet Protocol), HTTP (Hypertext Transfer Protocol), UDP (User Datagram Protocol), ICMP, POP (Post Office Protocol), FTP (File Transfer Protocol), and DCOM (Distributed Component Object Model) Any other known communication protocol or technology.

[0036] In some embodiments, the first microphone 1016 can be oriented toward the external environment side 150 or the sound source away from the wearer, and is used to capture the sound in front of the device 101, that is, to obtain the voice input of the communicating object or the communicating user, and the second microphone 1017 can be oriented toward the near-eye side 160 or the lower side of the frame or close to the temple 130 or the position close to the wearer's sound source, and is used to capture the user's voice input; in some embodiments, the first microphone 1016 uses an omnidirectional microphone or a MEMS microphone with higher sensitivity, which is placed on the temple or the top of the frame 120 to capture the voice of the external user (EU), and the second microphone 1017 can use a supercardioid or cardioid microphone, which is close to the user's mouth to capture the voice of the local user (LU); in other embodiments, the first microphone 1016 and the second microphone 1017 can also use a microphone array to form beamforming to obtain the voice input of the external user and the local user respectively. In other embodiments, the user's voice input and the voice input of the external object can also be obtained respectively by combining near-field and far-field audio processing algorithms or active noise cancellation.

[0037] In some embodiments, the first speaker 1018 can be located on the left temple 130, and the second speaker 1019 can be located on the right temple 130 to facilitate the voice-enabled playback function. In some embodiments, the optical sensor 1021 can be a brightness sensor, which can detect the state of the ambient light to adaptively change the display brightness of the microdisplay 1101, for example, when the external ambient light is bright, the display brightness of the microdisplay 1101 is increased, and when the external ambient light becomes dark, the display brightness of the microdisplay 1101 is adaptively reduced.

[0038] In some embodiments, the camera system 1025 may be a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) optical sensor, which may be used, for example, to record photos and videos. In one embodiment, the camera system 1025 may have a 180° FOV (field of view), and the camera system 1025 may be facing the environment side 150. The camera system 1025 may also include a depth sensor, which may include an infrared emitter that projects a point onto an object / subject, which is then photographed by a dedicated infrared camera and analyzed to determine depth data. In one embodiment, a time of flight (TOF) camera may be used to resolve distance based on the known speed of light and measuring the time of flight of the light signal between the camera and the object / subject for each point of the image.

[0039] In some embodiments, the position sensor 1020 may include a global navigation satellite system (GNSS) receiver, such as a global positioning system (GPS) or a Beidou positioning system, to determine the user's position, and the inertial sensor 1024 may include, for example, an accelerometer, a gyroscope, a magnetometer, etc. to provide information about the direction the user is facing or provide commands with head postures or gestures, etc. In one embodiment, the camera direction may be controlled by the head posture so that the camera system 1025 view follows the user's line of sight. The camera system 1025 may be located in the frame 120 and face the environment side 150.

[0040] In some embodiments, the biometric sensor 1022 can detect the wearer's temperature, fingerprint, facial recognition, heart rate and other health parameters. The biometric sensor 1022 can be located close to the user's temple wearing position or nose pad position; in some embodiments, a tactile motor (not shown) can also be included, which provides a vibration pattern to the user as tactile feedback.

[0041] In some embodiments, the battery 1013 can be a rechargeable battery and can be located in one of the temples 130; the electrical interface 1014 can be a USB (universal serial bus) interface or a metal contact or a wireless interface with magnetic attraction. The interface 1014 can be located on the temple 130 or the frame 120. As a USB interface, it can specifically be a Mini USB interface, a MicroUSB interface, or a USB Type C interface. The interface 1014 is used to connect a charger to perform wired charging on the battery 1013. In another embodiment, the interface is a wireless coil, for example, as a wireless receiver, to realize wireless charging of the battery. Of course, the interface can also be provided as a plurality of metal contacts or elastic contacts for wired charging of the battery. Of course, the electrical interface 1014 can also be used to transmit data or firmware upgrades between the device 101 and the external terminal device 103.

[0042] In some embodiments, the user interaction input 1023 may include a virtual button for displaying a graphical interface, or may include one or more physical buttons or touch sensors, such as touch capacitance. The interaction input 1023 may be located on the frame 120 or the temple 130; in other embodiments, the user interaction input 1023 may be provided as an interface for receiving interaction operations, and the actual user interaction operation may be located on the terminal device 103, and remote interaction input may be achieved through the application program interface of the terminal device 103. In other embodiments, the user interaction operation may also come from wearable devices such as smart rings, bracelets, or handles, which may be connected to the wearable device 101 in communication, such as a short-range Bluetooth connection, etc. The interaction input signal of the ring is sent to the user interaction input 1023, thereby achieving remote interaction operations.

[0043] It can be understood that the positions of the above-mentioned related sensors and other components can be arranged at different positions of the frame 120 and / or the temples 130 as needed; Figure 2 A schematic diagram of a structure for realizing content presentation by placing a display module 110 at a position different from that of an optical waveguide, which also includes the following Figure 4 The middle one is the sensor for realizing related functions; Figure 5 The figure in the figure is an example of presenting content in front of the lens or field of vision, and the position layout of each component is only a reference for implementing related functions.

[0044] A content translation method proposed in an embodiment of the present application includes:

[0045] S10, obtaining first content, where the first content includes at least a first language;

[0046] The first content may include one or a combination of images, videos, text, voice, etc. The first content may be a real-time text stream, voice stream, conversation stream or video stream, etc., and may be obtained by the wearable device 101 through its own image or audio unit, etc. For example, a user wearing the wearable device 101 communicates with an external user (external user, EU) through a microphone (such as the first microphone 1016 and the second microphone 1017). In other embodiments, the camera system 1025 is used to obtain picture content, voice or video stream of the communicating user (external user, EU), etc. Of course, the first content may also be obtained in real time from the terminal device 103 and then sent to the wearable translation device 101; in other embodiments, the first content may also come from the local storage of the wearable translation device 101 or the terminal device 103. In other embodiments, the first content may also be received by the communication unit 1015 from text, video conference or voice of an external terminal device or server. The first content includes at least the first language, for example, the picture includes the text of the first language, the video stream includes the text or voice or pronunciation of the first language, the audio stream includes the pronunciation of the first language, the text includes the vocabulary and field of the first language, etc. The first content may include all the first language. In other embodiments, the first content may also include part of the first language. The first language may be selected as any language, such as Chinese, English, Japanese, Korean, Spanish, etc. In some application scenarios, the first content may come from scenes such as communication and chatting, shopping, ordering, game voice or conference. For example, the wearer wears the wearable device to shop or order food, and obtains the picture information of the product through the camera system 1025. At this time, the first language is a language that the wearer has not mastered, and he cannot pronounce and use most of the practical vocabulary of the language. The wearer can actively select the category of the first language according to the location, scene, etc., and of course, the device can also automatically confirm the type of the first language by actively obtaining it through the microphone.

[0047] In some embodiments, the first language is the language mastered by the external user (EU) being communicated, which depends on the country, education and culture of the external user, etc. The native language of the external user (EU) can be preferably used as the second language. For example, if the external user is British or American, the first language is English; if the external user is Chinese, the first language is Chinese; if the external user is Korean, the first language is Korean, etc.

[0048] S20, converting the first content into second content by the first conversion, where the second content includes a second language, and the second language is different from the first language;

[0049] Among them, the first conversion includes a translation system, which can be a statistical machine translation (SMT), for example, a parallel corpus including a large number of words, phrases and grammatical rules, and translation is achieved based on the correspondence between a large number of multiple languages. The translation system can also be a neural machine translation (NMT), using recurrent neural networks (RNN), long short-term memory networks (LSTM), attention mechanisms (Attention Mechanism) and transformer models including self-attention mechanisms (Self-Attention Mechanism) and encoder-decoder architectures (Encoder-Decoder Architecture); in some embodiments, the API (Application Programming Interface) of external translation platforms such as Google Translate, Baidu Translate, Microsoft Translate, and Tencent Translate can be called. Interface, application programming interface) interface to achieve translation access, this translation platform corresponds to the external server 104; of course, the local neural network machine translation system built by the local server 105 can also be used to connect the wearable device 101 and the local server 105 to achieve the first content is sent to the local server 105 for translation; in some embodiments, the first conversion translation system can select existing large language models such as BLOOMZ (autoregressive model), GPT (Generative Pre-Trained Transformer, generative pre-trained model based on transformer architecture), ChatGLM (General Language Model, general language model), LLaMA (Large Language Model MetaAI, large basic language model), OPT (Open Pretrained Transformer, open pre-trained model), Claude model, Qwen (Tongyi Qianwen) or Wenxin Yiyan. After the first conversion translation system is determined, the server that processes the data will be fixed, for example, using the call of the corresponding database and instruction set based on this server.

[0050] In some embodiments, the first conversion may also include functions or systems such as image processing, machine recognition, speech recognition, audio-to-text, noise processing, optical character recognition (OCR), etc. The additional required functions of the first conversion depend on the input type of the first content. For example, if the first content input is a picture including a first language, the first language can be recognized and extracted through image processing combined with optical character recognition to obtain text including the first language, and then the first language text is sent to the translation server for translation into text in the second language.

[0051] The second content includes a second language different from the first language. The second language can be selected as any language, such as Chinese, English, Japanese, Korean, Spanish, etc. When the first language is Chinese, the second language is not Chinese. The second language is the language mastered by the local user (LU) wearing the wearable translation device 101, which depends on the country and education culture of the local user. The native language of the wearer user (LU) can be preferred as the second language. For example, if the wearer as a local user (LU) is Chinese, the second language is Chinese (Chinese); if the local user is Korean, the second language is Korean, etc. In some embodiments, a communication scenario example is taken in which a Chinese who masters Chinese is the wearer's local user and an American who masters English is the external user. The first content includes English voice, and the second content includes Chinese text obtained by translating English through the first conversion.

[0052] In some embodiments, the second content can maintain consistency with the file type of the first content. For example, the first content is a video stream, and the file of the second content is also a video stream, but it is changed from the first language to the second language after the first conversion, and can be presented through the display module 101; the first content is an audio stream, and the second file is also an audio stream, and the audio stream containing the first language is converted into an audio stream containing the second language after the first conversion; in some other embodiments, the first content is a text containing the first language, and the second content is a text containing the second language after the first conversion; in other embodiments, the file type of the second content is a text type, that is, the first content such as pictures, videos or audio containing the first language will be converted into a text type including the second language and presented, and can be presented through the display module 101, for example, finally projected onto the lens 140, or the micro display 1101 cooperates with the optical module 1108 and projects toward the human eye side 160, so that the second content including the second language can be seen by the local user (LU). In some embodiments, the file type of the second content may also include an audio type, that is, an audio type containing a second language may be generated based on the text type, and may be played through the first speaker 1018 and / or the second speaker 1019 and heard by a local user.

[0053] S30: Generate third content according to the second content, wherein the third content at least includes:

[0054] A reply text in reply to the second content, the reply text is composed of the second language; a translation text generated based on the reply text, the translation text is obtained by the first conversion, and the translation text is composed of the first language; a phonetic text based on the translation text, the phonetic text is obtained by a second conversion different from the first conversion, the phonetic text includes the second language and the pronunciation is the same as or similar to the first language.

[0055] The third content can be a real-time response from the local user (LU) wearing the wearable device 101. For example, the local user actively generates a response in the second language based on the second content. The user can input voice through the microphone of the wearable translation device 101 (such as the second microphone 1017) and then convert it into a reply text. Or the wearable translation device 101 sends the second content to a server with machine learning or a large language model to obtain a feedback reply to generate the third content. The user determines the input reply text through the user interaction input 1023 (such as swiping, confirmation button). Of course, it can also generate a reply text with a reply text based on certain specific template formats in combination with the user's active reply (using the second microphone 1017). It can be understood that after the local user receives the second content including the second language on the wearable translation device 101, for example, sees a question request in the second language on the display module 110 of the wearable translation device 101, or hears a question request in the second language through the wearable translation device 101, the local user can generate a reply text such as voice, text, or interaction according to their own scenarios, knowledge, experience, etc. The reply text can be generated by converting the voice of the local user into text by the processor of the wearable translation device 101, or it can be a reply text obtained by the user through interactive operations or supplementary voice replies in combination with the feedback of the large language model. The reply text is composed of the second language to ensure that it is the language input mastered by the local user.

[0056] The reply text can be further sent to the first conversion, that is, sent to the translation system. As a result, the reply text is translated into a translation text containing the first language that can be understood by the external user. At this time, the wearable translation device is still worn by the local user, and the external user cannot directly see this reply text and the associated translation text. The translation text will be processed by the second conversion to obtain the phonetic text. The second conversion is different from the first conversion. It can be understood that the second conversion does not follow the standard vocabulary, phrases, and grammar rules of the parallel corpus, but is based on the rule that the pronunciation of the second language is the same or similar to that of the first language. It can be understood that the database of the second conversion is different from that of the first conversion, and the data processing rules of the second conversion and the first conversion data processing rules are also different. The phonetic text can be obtained by looking up the table method based on the same or similar pronunciation between languages. In some embodiments, for example, the reply text contains the Chinese "非常感谢" in the second language, and the translation text is translated into "Thank you very much" in English in the first language through the first conversion. The first conversion follows the standard language translation rules, while the phonetic text obtained through the second conversion is "三克 优 歪瑞骂吃" in the second language (Chinese). It can be seen that the second conversion does not follow the standard language translation rules, and its database construction rule only maintains the same or similarity in pronunciation.

[0057] S40, presenting the translated text and the phonetic notation text.

[0058] It can be understood that the translation text composed of the first language and the phonetic text composed of the second language will be presented, for example, displayed on the display module 110 of the wearable translation device 101, and the local user can watch them at the same time from the near-eye side 160, but the external user can not see the third content containing the translation text and the phonetic text without wearing the wearable translation device 101 (unless the local user gives the wearable translation device to the outside for wearing), so the local user can read out or read aloud the corresponding pronunciation based on the presented translation text of the first language and the phonetic text of the second language. Therefore, in the case where the local user is not proficient in the first language, for example, the local user is not clear and masters the accurate pronunciation of the words, phrases or sentences in the first language, the local user uses the second language that he masters to express the translation content of the first language through his own pronunciation, that is, the local user uses the mastered native language to say the information content that he wants to convey to the external user (the other party). Therefore, this application can effectively improve the communication efficiency between local users and external users, and does not require external users to wear additional head-mounted devices or similar translation devices, shortening the distance between people and reducing the social disadvantages of local users due to language. On the other hand, external users do not need to use the device of the local user to contact the skin, effectively avoiding the situation where the same device is used alternately and repeatedly to view the content expressed by both parties. In addition, the reply text, translation text and phonetic text can only be seen by the local user, and are not easily seen by third-party users, which effectively ensures the privacy of the communication content of both parties. At the same time, the local user can view it through the near-eye side 160 of the wearable device 101 during the communication process, for example Figure 2 In the example, the third content including the translated text and the phonetic text is presented in the frame. When the local user views the translated text and the phonetic text, the display content of the display module 101 does not interfere with the physical world. The local user and the external user can maintain eye contact during the communication process without having to repeatedly watch the translated content on the translation software on terminal devices such as mobile phones, thereby shortening the emotional distance between people.

[0059] In some embodiments, the second content including the second language, the reply text, the translation text and the phonetic text are presented in sequence on a display associated with the wearable device according to a triggering order.

[0060] Combined with reference Figure 1-Figure 10 The wearable translation device 101, the external terminal device 103 and the servers 104 and 105 jointly establish a communication connection. The wearable device 101 is provided with a first display area 1200. The first display area 1200 can be located at Figure 5 At the lens 140 position or Figure 2At the position of the frame 120 in the terminal device 103, a second display area 1300 is provided, wherein the display area of ​​the second display area 1300 is larger than the display area of ​​the first display area 1200, that is, the second display area 1300 can present significantly more content than the first display area 1200 in the same time period.

[0061] Continue reading Figure 7, taking a data flow as an example, the wearable translation device 101 receives a first content 170 input including a first language from an external user (EU), and then sends it to the terminal 103 through the communication unit, and the terminal 103 continues to send the first content to the server 104. In some embodiments, the second display area 1300 of the terminal 103 presents the first content 170 including the first language (for example, English); the server 104 translates the first content 170 into the second content 171 including the second language (Chinese) through the first conversion, and the second content 171 is returned to the terminal 103, and the second content 171 is displayed on the second display area 1300 of the terminal 103. The second content 171 is presented in the second display area 1300, and the second content 171 is continuously sent to the translation device 101. After receiving the second content 171 in the second language, the translation device 101 can present the second content 171 in the first display area 1200. The local user can quickly view the second content 171 on the translation device 101 and then reply. The translation device 101 receives the reply from the local user (LU) to form a reply text 173. The reply text 173 can be presented in the first display area 1200. After the reply text 173 is sent to the terminal 103, it can be presented in the second display area 1300 of the terminal 103. The reply text 173 is sent to the server 104 (105) and then generates a translation text 174 through a first conversion. The translation text 174 continues to generate a phonetic text 175 through a second conversion different from the first conversion. The translation text 174 and the phonetic text 175 are sent to the terminal 103 again. The translation text 174 and the phonetic text 175 can be presented (displayed) in the second display area 1300. Finally, the translation text 174 and the phonetic text 175 are sent to the translation device 101 and presented in the first display area 1200. Thus, the first content 170, the second content 171, the reply text 173, and the translation text 174 are displayed. The second content 171, the translation text 174 and the phonetic text 175 are displayed in the second display area 1300 from top to bottom according to the trigger sequence, and the area of ​​the first display area 1200 is smaller than that of the first display area 1200. To provide higher communication display efficiency, the first display area 1200 is triggered to display the second content 171, the translation text 174 and the phonetic text 175 from top to bottom. In another embodiment, to further improve communication efficiency, the first display area 1200 also displays the second content 171, the reply text 173, the translation text 174 and the phonetic text 175 from top to bottom according to the trigger sequence.The present application omits the presentation of the first content 170 in the first language on the first display area 1200 and directly presents the second content in the second language, which effectively saves the local data processing time of the translation device 101 and directly presents the content of the external user's communication. Therefore, the local user can directly express the content of the translated text with the same or similar pronunciation in the second language he or she masters, that is, the local user uses his or her own mouth to express the language he or she is not proficient in, which effectively improves the efficiency of the local user in receiving information. In other embodiments, the reply text 173 and / or the second content 171 may not be presented on the first display area 1200, further reducing the data processing time.

[0062] In some application scenarios, the data processing and storage capacity of the terminal device 103 is greater than that of the wearable translation device 101, for example, it has a larger memory and processor. In order to facilitate users to view or read historical communication data or synchronized communication content, the local user (LU) can view and retrieve all data of the entire communication process through all content data presented by the terminal device 103. In some embodiments, this content data can be synchronized in real time, and only part of the content data is presented on the translation device 101, and only effective communication data can be seen, thereby reducing data redundancy in the presentation of inefficient communication content, thereby improving data processing efficiency, reducing hardware consumption such as memory or processor of the wearable device 101, and reducing the hardware cost of the wearable device 101.

[0063] In some embodiments, in combination with Figure 6 , the wearable translation device 101 and the servers 104 and 105 directly establish a communication connection, that is, the communication connection with the terminal device 103 is reduced. The wearable translation device 101 receives the fixed first content 170 in the first language input by the external user (EU) and directly sends it to the servers 104 and 105. The servers 104 and 105 can be the existing commercial servers or locally built servers described in the above embodiments. The second content 171 is generated by the first conversion in the servers 104 and 105, and the second content 171 is then sent to the translation device 101. In some embodiments, such as Fig. 9As shown, the second content 171 can be first presented on the first display area 1200 of the translation device 101, and the translation device 101 receives the reply text 173 of the local user again. In some embodiments, the reply text 173 can also be presented on the first display area 1200, and in other embodiments, the reply text 173 and / or the second content 171 may not be presented on the first display area 1200; the reply text 173 is sent to the server 104, 105, and then the first conversion is performed to generate the translation text 174, and the translation text is converted into the phonetic text 175 through the second conversion, and finally the translation text 174 and the phonetic text 175 are sent to the translation device 101 and then sequentially displayed in the first display area 1200, and the local user can directly use the second language mastered to express the content of the translation text containing the first language with the same or similar pronunciation, that is, the local user uses his own mouth to personally express the language that he is not proficient in and communicates with the other party, which effectively improves the communication efficiency. When the local user or external user wants to retrieve comprehensive historical communication data, it can be viewed on the external terminal device 103.

[0064] In some embodiments, the presentation delay of the translated text 174 and the phonetic text 175 in the first display area 1200 is greater than the presentation delay of the translated text 174 and the phonetic text 175 in the second display area 1300, that is, the translated text 174 and the phonetic text 175 are presented first on the terminal device 103, and the translated text 174 and the phonetic text 175 are presented second on the first display area 1200 of the translation device 101, thereby giving sufficient time for the local user's eyes to adapt, so as to accurately and comprehensively see the presented text content.

[0065] In some embodiments, see Figure 9-10, the second display area 1300 includes a first prompt 183 and a second prompt 184, the first prompt 183 is associated with the first content 170 and the second content 171, and the second prompt 184 is associated with the third content 172, wherein the first prompt 183 may be a user identifier (for example, the external user is EU) located in front of the first content 170 and the second content 171, wherein the user identifier may include multiple identical or different external users, the first prompt 183 may also include an envelope, background color or font surrounding the first content 170 and the second content 171, for example, it may be an envelope composed of points, and the second prompt 184 may be a user identifier (for example, a local user is LU) located in front of the third content 172, wherein the user identifier may include multiple identical or different local users, the second prompt 184 may also include an envelope, background color or font, etc. surrounding the second content 171, for example, an envelope consisting of dot-dot or dot-line, the first prompt 183 and the second prompt 184 may be different; the first display area 1200 includes a third prompt 185 and a fourth prompt 186, the third prompt 185 is associated with the second content 171, the fourth prompt 186 is associated with the third content 172, and the third prompt 185 may be located The user identifier in front of the second content 171 (for example, the external user is EU), wherein the user identifier may include multiple identical or different external users, the third prompt 185 may also include an envelope, background color or font, etc. surrounding the second content 171, for example, it may be an envelope composed of dots. The fourth prompt 186 may be a user identifier in front of the third content 172 (for example, the local user is LU), wherein the user identifier may include multiple identical or different local users, the fourth prompt 186 may also include an envelope, background color or font, etc. surrounding the third content 172, for example, it may be an envelope composed of dots or dots. Among them, the first prompt 183 and the third prompt 185 can be the same, and the second prompt 184 and the fourth prompt 186 can be the same, for example, having the same user character expression, background color, or font, etc.; in other embodiments, the first prompt 183, the second prompt 184, the third prompt 185 and the fourth prompt 186 can also include a group or combination of the color, font, background color or prompt letters or fields of the corresponding content. The above method can improve the user's efficient viewing of the content in the communication process on different device terminals. The same content has the same identifier can reduce the probability of misreading the content and improve the reading accuracy of different content presentations.

[0066] In some other embodiments, the first display area 1200 may further include user interaction indicators such as read aloud 181 and a drop-down bar 182 , wherein the read aloud 181 may be used to indicate an operation of reading aloud the standard pronunciation of the translated text 174 .

[0067] In some embodiments, such as Fig.10 As shown, the first content 170, the second content 171, the reply text 173, the translation text 174 and the phonetic text 175 may appear periodically according to a time trigger sequence, and the amount and sequence of each content presentation depends on the actual communication or application scenario between the current local user LU and the external user EU.

[0068] In some embodiments, the presentation speed of the translated text 174 and the phonetic text 175 in the first display area 1200 is lower than the presentation speed of the translated text 174 and the phonetic text 175 in the second display area 1300. It is understandable that the blinking frequency of the human eye and the information that can be read within a certain period of time are limited. Since the display area of ​​the first display area 1200 is small, if it is presented at the same speed as the second display area 1300 or at a faster speed, it will make it difficult for the local user to clearly see the text content presented in the first display area 1200. Therefore, it is necessary to adaptively reduce the presentation speed of the translated text 174 and the phonetic text 175 in the first display area 1200, for example, present them at an interval speed of seconds.

[0069] In some embodiments, the first content includes one or a combination of text, image, video, and voice in a first language obtained by the wearable translation device 101 from the real environment, and the second content includes one or a combination of text, image, video, and voice in a second language returned after being processed by an external server. Fig.11, taking a scenario data service example of a certain meal ordering, the wearable translation device 101 obtains a picture of an English menu list (meallist) through the camera system 1025 as the first content input. The picture of the menu list is sent to the terminal or the server for image-to-text and translation conversion to generate the second content of the Chinese menu list, which is presented in the display area of the wearable device 101. The user can make a selection according to their own preferences and then confirm the order to form the reply text 173 of "chicken salad". This reply text 173 is sent to the external server for the first conversion to generate the translation text 174 of "I would like to order the chicken sala, please". Subsequently, based on the translation text 174, a phonetic text 175 including Chinese "艾 伍德 来克 吐 哦德尔 的 曲肯 萨拉,破历史" is generated through the second conversion. Local users can use the presented translation text and phonetic text for pronunciation expression to external users (such as waiters). After the waiter's English reply, it will continue to be converted to Chinese translation through the first conversion and presented on the wearable translation device. The user can actively reply with Chinese "非常感谢", or in some other embodiments, this reply text includes one or more reference reply texts or contents automatically generated based on AI (Artificial Intelligence), large language models, machine learning models, etc. For example, the server side can give one or more reference reply texts based on the context of the situation. The user can select and confirm to reply with a certain reference reply content or text. Subsequently, the reference reply content continues to be converted to the translation text 174 of "Thank you very much" and the phonetic text of "三克 优歪瑞骂吃" through the first conversion. Thus, local users can quickly communicate and reply without much thinking and read out in the language they master.

[0070] In some embodiments, a real-time voice stream of a user is obtained, for example, a real-time voice stream of an external user and / or a real-time voice stream of a local user is obtained through a microphone on the translation device 101 (for example, the first and second microphones 1017), and the voice stream is sent to a large language model or a natural language server; the sound characteristics of the voice stream are analyzed to obtain the emotional state of the external user; the mood or situation of the external user can be confirmed through the tone, intonation and speed in the voice stream. For example, when the tone, intonation or speed is very fast, it can be confirmed that the external user is very excited. In another embodiment, the emotional state can be obtained by obtaining the facial expression of the external user through the camera system 1025, and a plurality of reference reply texts are generated based on the emotional state, wherein each reference text is generated based on the contextual situation. For example, when the emotional state is angry, a flattering text or a caring text is provided. Taking Chinese as an example, the reference text can be For example, "Baby, don't be angry", "Your clothes and shoes are so beautiful today", etc., when the emotional state is disappointment, encouraging text is provided, such as the example of "Persistence is victory" in Chinese as the second language, etc. Each reference reply text is associated with a corresponding translation text and phonetic text. The server 104 can provide a translation text in the first language and a phonetic text in the second language for the reference reply text. In other embodiments, multiple reference reply texts can also be provided first, and the corresponding translation text and phonetic text will be provided after the local user confirms that one of the reference reply texts has been selected. It can be understood that the current emotion is excited and needs to be comforted or the situation is very urgent, so a reference reply text with a soothing emotion and a pleasing personality will be generated, and the corresponding translation text and phonetic text will continue to be generated based on this reference reply text, thereby facilitating cross-language emotional communication and shortening the distance between people.

[0071] In some embodiments, after the local user wears the translation device 101, the location sensor 1020 on the device 101 can obtain the location of the local user. For example, if the local user is in a shopping store, it is determined that the user is in an offline shopping scene, and if the local user is in a train station or a bus station, it is determined to be a travel scene; or the camera system 1025 on the device 101 can obtain information about a large number of books in the environment where the local user is located, thereby determining it as a library scene, or the camera system 1025 obtains information such as text and graphics on the computer screen and cooperates with GPS, and the sensor confirms it as an office scene; thus, reference reply texts in different scenes can be constructed according to the location of the local user. In some embodiments, the content translation method may also include obtaining a fourth content input from the local user, and the fourth content includes a second language, such as Fig.12In the example, the location sensor 1020 obtains that the local user is in a place he has never been to before, and the local user wearing the translation device 101 is looking for a meal on the street. At this time, the local user can actively propose a fourth content composed of a second language (such as Chinese), such as "how to ask the foreigner next to you where to have lunch". At this time, the translation device sends the demand to the large model of the server 104 and 105 for processing. The server generates a first reference reply text and a corresponding translation text, a second reference reply text and a corresponding translation text. The above texts are sent to the translation device 101 and presented. The user may feel that the first reference reply text and the corresponding translation text are suitable for his expression based on the presented content, so he selects the first reference reply text. Reply text 173, and then the translated text in the first reference reply text continues to be sent to the server to generate phonetic text 175 through a second conversion. In some other embodiments, only the reference reply text in the second language can be presented first. For example, the translated text in the second language is presented as "Excuse me, do you know where I can have lunch nearby?" Then, based on the reference reply text selected by the user, a translated text 174 is generated through the first conversion, and the translated text 174 continues to generate phonetic text 175 through the second conversion. Therefore, the content translation method of the present application can generate a reference reply text that is more matched and in line with the scene based on the active needs and scenes of the local users for reference by the local users, saving the local users' time for active thinking and improving communication efficiency.

[0072] In some embodiments, generating the third content based on the second content may also include obtaining the text form of the second content, which may include closed content, open content statements or polite content; generating a reference reply text based on the text form, taking the second language as Chinese as an example, for example, closed content may include questions composed of core keywords such as "really", "do you want it", "is it okay", etc., and for such closed content, short reference reply texts such as "yes", "no", "don't want it", "yes", "ok", "no" may be provided; for example, open content may include development questions such as "how", "how", "why", "what is it", etc., and clear, specific and structured grammar or logic such as "I think it is very good, first aspect, second aspect..." may be provided for reply. Fig.11 , and polite expressions such as "thank you very much" and "you're welcome" can also be provided for statement contents with service logic. In some embodiments, the server 104 or the terminal device 103 can store a large amount of latest reference reply texts or external updates. This reference reply text corresponds to each second content in different text forms, thereby facilitating efficient communication for local users and reducing the thinking time required for replies.

[0073] In some embodiments, the reply text in response to the second content also includes: receiving a real-time voice stream of a local user wearing the wearable translation device 101, converting the local user's voice stream into a reply text, and associated translation text and phonetic text, that is, obtaining the local user's real-time voice stream reply through the second microphone 1017, sending the voice stream data to the terminal device 103 or the server 104, 105 to convert the audio content into text content, and then translating the text content to obtain a translation text including the second language, and then generating a phonetic text including the first language based on the translation text of the second language by the second conversion.

[0074] In some embodiments, the content translation method also includes: determining the voiceprint registration of the user wearing the wearable device; when the user wears the wearable device, an audio signal of a local user reading a preset short sentence (such as hello, I am your personal assistant) can be obtained through a microphone (such as the second microphone 1017) and then converted into a digital signal, and then pre-processed such as denoising and emphasis, and then features such as spectrum, channel or frequency are extracted from the signal, such features are combined to generate a user model, and the model is stored in a database for identifying and verifying the local user. When the local user and the external user generate voice streams, the voiceprint registration information can be used to separate the local user's voice and the external user's voice, thereby improving the accuracy of the user's voice stream input during the communication process.

[0075] Obtaining the pronunciation of the local user for the phonetic text; the pronunciation of the local user (LU) can be received by the second microphone 1017 on the wearable translation device 101. In other embodiments, the pronunciation can also be received by a microphone (not shown) on the terminal device 103, thereby reducing data transmission time and improving data processing speed;

[0076] The user's pronunciation is compared with the standard pronunciation of the translated text; the standard pronunciation of the translated text can come from an existing translation database. For example, the user can choose the standard pronunciation corresponding to the first language, such as British pronunciation or American pronunciation for English. For other languages, a custom standard pronunciation can be selected based on the database.

[0077] Combined with reference Fig. 9, provide accuracy and present accuracy 180 according to the comparison result. When the local user reads the translated text in the first language according to the phonetic text in the second language, the whole pronunciation can be stored and compared according to the tone, intonation and pronunciation syllables. For example, the above features can be weighted and normalized, and compared with the weights of the standard pronunciation database. The similarity values ​​under different weights can be divided. The closer the weight values ​​of the comparison between the two are, the higher the similarity and the higher the pronunciation accuracy. In other embodiments, the facial expression features of the external user can also be added as comparison weights. For example, the camera system 1025 can obtain the facial expression of the external user. The facial expression of the external user can be visualized as normal, confused or happy through machine learning or deep image processing. The weight of the confused expression is low, indicating that the external user does not really understand the pronunciation of the local user. The weight of the happy expression is high, indicating that the external user recognizes the pronunciation of the local user. The weight of the normal expressionless is the median, indicating that the external user understands the pronunciation of the local user. In other embodiments, the local user may also proactively identify text content that is not accurately pronounced, and store it on the external terminal device 103, so that it can be retrieved next time and practiced or reviewed on the external terminal device 103 or the translation device 101. By synchronously presenting the accuracy 180 of the local user's pronunciation of the phonetic text 175 in the first display area 1200, the communication content can be fed back to the local user in real time, so that the local user can adjust or re-express the text content in real time.

[0078] In some embodiments, the content translation method may further include presenting the standard pronunciation below the translated text. The standard pronunciation may be the international standard pronunciation. In some embodiments, the method may further include annotating the phonetic text with pronunciation, and providing a pronunciation prompt for the pronunciation accuracy that does not meet the expected value, such as a color (such as red or yellow) marking reminder. In other embodiments, the standard pronunciation may be presented below the translated text 174, or the key words and phrases of the phonetic text may be marked with color, letters, or tones to remind the local user of the correct pronunciation of a specific text and improve communication efficiency.

[0079] The present application also provides a wearable translation device 101, which can be used in conjunction with Figure 1-Figure 5 The device includes a shell, which may be the frame 120 or the temple 130 in the above-mentioned embodiment, at least one processor located in the shell, for example, located in the temple 130, and a memory storing instructions, wherein the processor is configured to execute the instructions stored in the memory, and when the instructions are executed by the processor, the processor is configured to execute the content translation method provided in the above-mentioned embodiment.

[0080] The present application also provides a content translation system 300. Fig.13, the system 300 includes one or more processors 301 (for example, a single-core or dual-core processor), a network interface 304, one or more memories 302, and a communication bus 303. The communication bus 303 can utilize various hardware and software to facilitate the transmission of data and control signals between components. The network interface 304 uses any transceiver-like device for communicating with other devices or communication networks, and the communication network can be Ethernet, a radio access network (RAN), or a wireless local area network (WLAN). The network interface 304 can include a wired communication interface and can also include a wireless communication interface. Specifically, the network interface 304 can be an Ethernet interface, a Fast Ethernet (FE) interface, a Gigabit Ethernet (GE) interface, an Asynchronous Transfer Mode (ATM) interface, a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The memory 302 is, for example, a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 320 is, for example, independent and connected to the processor 301 via the bus 303. The memory 302 can also be integrated with the processor 301. The system 300 can be included in any computer device, which includes one or more server computers in a local network or a distributed network, each server computer having one or more processing cores. System 300 may be implemented in a parallel processing or peer-to-peer infrastructure or on a single device having one or more processors. Memory 302 stores program instructions that, when executed by processor 301, perform the functions described herein, including methods associated with any of the above-described content translation method embodiments.

[0081] In an embodiment of the present application, a computer-readable storage medium is also provided. The computer-readable storage medium stores computer instructions. When the computer instructions stored in the computer-readable storage medium are executed by a computer device, the computer device executes the content translation method provided in the above embodiment.

[0082] In an embodiment of the present application, a computer program product including instructions is also provided. When the computer program product is executed on a computer device, the computer device executes the content translation method provided in the above embodiment.

[0083] To sum up, the solution proposed in this application can effectively convert and communicate between the two languages ​​of the user, is suitable for cross-language communication scenarios, can help users understand and express information more easily, and provide homophonic text corresponding to the translated content in the original language to help users more easily master the correct pronunciation and intonation when expressing, thereby increasing user interactivity and participation.

[0084] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0085] The above are only optional embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed in the present application should be covered by the protection scope of the present application.

Claims

1. A content translation method, characterized in that: The method comprises: Acquire first content, where the first content includes at least a first language; Converting the first content into second content by a first conversion, wherein the second content includes a second language, and the second language is different from the first language; Generating third content according to the second content, wherein the third content at least includes: a reply text in response to the second content, the reply text being in the second language; A translation text generated based on the reply text, the translation text being obtained by the first conversion and being in the first language; A phonetic text based on the translated text, the phonetic text being obtained by a second conversion different from the first conversion, the phonetic text including the second language and having a pronunciation identical to or similar to that of the first language; Presenting the translated text and the phonetic text; The acquiring of the first content comprises: Get the user's real-time voice stream; The generating second content from the first content by the first conversion further comprises: Sending the speech stream to a large language model or a natural language server; Analyzing the sound characteristics of the voice stream to obtain the emotional state of the external user; Generating the third content according to the second content includes: Generate multiple reference reply texts based on the emotional state, each reference reply text is associated with a corresponding translation text and phonetic text; The method further comprises: The wearable device establishes a communication connection with an external terminal device, the wearable device includes a first display area, and the external terminal device includes a second display area. The display area of ​​the first display area is smaller than the display area of ​​the second display area. The first display area at least presents the translated text and the phonetic text, and the second display area presents the first content, the second content, the reply text, the translated text, and the phonetic text.

2. The content translation method according to claim 1, characterized in that: The method further comprises: presenting the second content in the second language, the reply text, the translation text and the phonetic text in the second display area in sequence according to a triggering order; The presentation delay of the translated text and the phonetic text in the first display area is greater than the presentation delay of the translated text and the phonetic text in the second display area.

3. The content translation method according to claim 1, characterized in that: The first content includes one or a combination of text, image, video, and voice in a first language obtained by the wearable device from a real environment, and the second content includes one or a combination of text, image, video, and voice in a second language returned after being processed by an external server.

4. The content translation method according to claim 1, characterized in that: The method further comprises: A real-time voice stream of an external user and / or a real-time voice stream of a local user is obtained through a microphone. The wearable device includes a first microphone and a second microphone. The first microphone is used to capture the voice of the external user, and the second microphone is used to capture the voice of the local user.

5. The content translation method according to claim 4, characterized in that: The method further comprises: The second display area includes a first prompt and a second prompt, the first prompt is associated with the first content and the second content, and the second prompt is associated with the third content; The first display area includes a third prompt and a fourth prompt, the third prompt is associated with the second content, the fourth prompt is associated with the third content, the first prompt is the same as the third prompt, and the second prompt is the same as the fourth prompt; The presentation speed of the translated text and the phonetic text in the first display area is lower than the presentation speed of the translated text and the phonetic text in the second display area.

6. The content translation method according to claim 1, characterized in that: The method further comprises: Determine the voiceprint registration of the user wearing the wearable device; Obtaining a local user's pronunciation of the phonetic text; Compare the pronunciation of the local user's phonetic text with the standard pronunciation of the translated text; The accuracy is provided and presented based on the comparison results.

7. The content translation method according to claim 6, characterized in that: The method further comprises: Pronunciation marking of the phonetic text; Providing a standard pronunciation reading of the translated text; Pronunciation prompts will be given if pronunciation accuracy does not meet expectations.

8. The content translation method according to claim 1, characterized in that: The method further includes: obtaining the first content includes obtaining a real-time voice stream input from an external user, converting the first content into the second content includes converting the voice stream input into a translated text, A plurality of reference reply texts in the first language are generated based on the translated text, so as to display both the translated text and the reference reply text to the user.

9. A content translation system, characterized in that: include one or more processors, A memory storing program instructions, wherein the program instructions can be executed by the one or more processors, and the operations include: Acquire first content, where the first content includes at least a first language; Converting the first content into second content by a first conversion, wherein the second content includes a second language, and the second language is different from the first language; Generating third content according to the second content, wherein the third content at least includes: a reply text in response to the second content, the reply text being in the second language; A translation text generated based on the reply text, the translation text being obtained by the first conversion and being in the first language; A phonetic text based on the translated text, the phonetic text being obtained by a second conversion different from the first conversion, the phonetic text including the second language and having a pronunciation identical to or similar to that of the first language; Presenting the translated text and the phonetic text on a display module; The acquiring of the first content comprises: Get the user's real-time voice stream; The generating second content from the first content by the first conversion further comprises: Sending the speech stream to a large language model or a natural language server; Analyzing the sound characteristics of the voice stream to obtain the emotional state of the external user; Generating the third content according to the second content includes: Generate multiple reference reply texts based on the emotional state, each reference reply text is associated with a corresponding translation text and phonetic text; The wearable device establishes a communication connection with an external terminal device, the wearable device includes a first display area, and the external terminal device includes a second display area. The display area of ​​the first display area is smaller than the display area of ​​the second display area. The first display area at least presents the translated text and the phonetic text, and the second display area presents the first content, the second content, the reply text, the translated text, and the phonetic text.

10. A wearable translation device, characterized in that: include: case; at least one processor located in the housing; a memory storing instructions, the processor being configured to execute the instructions stored in the memory, the instructions, when executed by the processor, configuring the processor to: Acquire first content, where the first content includes at least a first language; Converting the first content into second content by a first conversion, wherein the second content includes a second language, and the second language is different from the first language; Generating third content according to the second content, wherein the third content at least includes: a reply text in response to the second content, the reply text being in the second language; A translation text generated based on the reply text, the translation text being obtained by the first conversion and being in the first language; A phonetic text based on the translated text, the phonetic text being obtained by a second conversion different from the first conversion, the phonetic text including the second language and having a pronunciation identical to or similar to that of the first language; Presenting the translated text and the phonetic text on a display module; The acquiring of the first content comprises: Get the user's real-time voice stream; The generating second content from the first content by the first conversion further comprises: Sending the speech stream to a large language model or a natural language server; Analyzing the sound characteristics of the voice stream to obtain the emotional state of the external user; Generating the third content according to the second content includes: Generate multiple reference reply texts based on the emotional state, each reference reply text is associated with a corresponding translation text and phonetic text; The wearable device establishes a communication connection with an external terminal device, the wearable device includes a first display area, and the external terminal device includes a second display area. The display area of ​​the first display area is smaller than the display area of ​​the second display area. The first display area at least presents the translated text and the phonetic text, and the second display area presents the first content, the second content, the reply text, the translated text, and the phonetic text.

Images