Multilingual simultaneous translation intelligent robot and method thereof
By combining positioning, visual and auditory sensors, the geographical location, appearance attributes and audio characteristics of the speech object are obtained, and the problem of difficulty in identifying multilingual translation robots in the prior art is solved, and fast and accurate language recognition and translation output are achieved.
Patent Information
- Application Number
- CN202411916523.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-24
- Publication Date
- 2025-04-08
AI Technical Summary
Existing multilingual translation robots find it difficult to quickly and accurately identify the language of the speaker in complex language environments, especially in noisy environments and multilingual coexistence areas, and there are problems such as difficulty in recognition and high computational cost.
By combining positioning sensors, vision sensors and auditory sensors, the geographical location, appearance attributes and audio characteristics of the speech object are obtained, the language is judged, and the target translation language is determined based on the audio characteristics.
It improves the recognition speed and accuracy of speech languages, reduces the language calculation and output time required for translation, and can accurately output key translation content in noisy environments.
Smart Images
Figure CN119783694A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent translation, and in particular to a multi-language simultaneous translation intelligent robot and a method thereof. Background Art
[0002] The multilingual translation robot is a system that integrates artificial intelligence and natural language processing technology, which can perform two-way translation between multiple languages and support text and voice input. This can provide users with multilingual translation services, greatly promoting barrier-free communication across languages and cultures.
[0003] In complex environments where multiple languages coexist, some service agencies have deployed multiple multilingual translation robots in order to facilitate the service of users. However, the current translation robots either need users to directly input the language to be translated before they can translate, or extract keywords and key words from the user's audio information to determine the user's language, and then determine the language that the user needs to translate based on voice recognition before they can perform relevant translations. However, this will result in the following situations: first, the user needs to be in a relatively quiet environment to recognize the user's voice; second, there are many languages currently, and the robot may only be able to recognize the user's language based on the user's long-term speech. At the same time, the accents of users in different regions are relatively different. The robot not only needs to recognize the speaker's audio, but also needs to find the language that his audio information matches, which increases the difficulty of robot recognition. Summary of the invention
[0004] The present invention aims to provide a multi-language simultaneous translation intelligent robot and method thereof, which can not only improve the recognition rate of the speaking language of the speaking user, but also improve the recognition speed of the speaking language, reduce the calculation amount of the language required for translation, improve the output speed of the translated language, and can accurately output the key translation content of the speaking user in a relatively noisy environment and with a small amount of calculation.
[0005] In order to achieve the above object, the present invention adopts the following technical scheme:
[0006] The first scheme: a multi-language simultaneous translation intelligent robot, used to translate the language of a speaking object and transmit it to a translation object, comprising: a first acquisition module, used to acquire the geographical location characteristics of the speaking object according to a positioning sensor, and acquire the languages existing in the geographical location according to the geographical location characteristics, wherein the existing languages include common languages and minority languages; a second acquisition module, used to acquire the appearance attribute characteristics of the speaking object according to a visual sensor, and acquire the language used by the speaking object according to the appearance attribute characteristics; a third acquisition module, used to acquire the audio characteristics of the speaking object according to an auditory sensor, and acquire the language used by the speaking object according to the audio characteristics; a first judgment module, used to judge whether the language acquired in the second acquisition module is within the range of the languages acquired in the first acquisition module; if it is within the range of the languages acquired in the first acquisition module, the language acquired in the second acquisition module is used as the speaking language of the speaking object; otherwise, the language acquired in the third acquisition module is used as the speaking language of the speaking object; a fourth acquisition module, after determining the speaking language of the speaking object, acquires the target translation language required for translation of the speaking object according to the audio characteristics.
[0007] Preferably, the minority languages include Tibetan, Urdu, Uyghur, Kazakh, Kirgiz, Xibe, Tajik, Kyrgyz, Iranian, Georgian and Uzbek.
[0008] Beneficial effects: Ability to translate multiple different minority languages, promoting barrier-free communication across languages and cultures.
[0009] Preferably, the appearance attribute features include facial features, clothing, accessories and the eye gaze direction of the speaker.
[0010] Beneficial effect: In a complex regional environment where multiple languages coexist, the appearance attributes of the speaker can be used as an important basis for judging the race of the speaker, and the race of the speaker can be judged through the intuitive appearance features of the speaker, thereby indirectly identifying the speaking language of the speaker. Not only is the recognition speed fast, but the features are more numerous and intuitive, thereby improving the accuracy of recognition.
[0011] Preferably, the audio features include timbre, loudness, rhythm, emotional color, voice clarity and speech information; wherein the speech information includes key words and key phrases.
[0012] Beneficial effect: The audio information of the speaker can be obtained based on the audio features, which can be used for accurate translation.
[0013] Preferably, it also includes: a language translation quantity judgment module, which executes a target confirmation module if there are at least three speakers and at least three target translation languages; a target confirmation module, which corresponds each speaking language to a different speaker according to the appearance attribute characteristics of the speaker and the audio characteristics of the speaker; a first recognition module, which obtains the eye gaze direction of the speaker according to the visual sensor and uses the eye gaze direction as the first visual data; a second recognition module, which obtains the personal keywords and the number of mentions of the personal keywords in the speech information of the speaker according to the auditory sensor, and uses the personal keywords and the number of mentions of the personal keywords as the first audio data; a translation language recognition module, which is used to use at least one of the first visual data and the first audio data as the first input data, and input the first input data into a first recognition model that has been trained; the first recognition model is used to output at least one target translation language to multiple target translation objects.
[0014] Beneficial effect: When there are at least three speakers and at least three target translation languages need to be translated at the same time, in the existing multilingual translation robot, only two-way translation is performed between multiple languages. Therefore, the existing multilingual robot either cannot directly output two target translation languages synchronously, or directly outputs all target translation languages, which not only greatly improves the translation volume of the robot, but also increases the translation time of the robot. At the same time, the speaker may only want to translate the dialogue object in the speech content, not necessarily translate other objects, so that the conversation content is leaked to other objects, and the privacy protection effect cannot be achieved. The target confirmation module confirms the speech language used by each speaker, and then obtains the relevant data of the eye sight direction of the speaker according to the first recognition module, as well as the personal keywords and the number of mentions of the personal keywords in the speech information of the speaker as the first audio data, and combines the first visual data with the first audio data, and the audio-visual combination is used as input data to output at least one target translation language through the first recognition model. In this way, the speaker's dialogue partner can be accurately determined based on the speaker's visual orientation and the personal keywords in the speech information, so as to accurately output the target translation language corresponding to the language used by the dialogue partner. By accurately understanding the speaker's dialogue partner, not only can the robot's translation volume and translation time be reduced, but the protection of the speaker's speech content and privacy can also be improved.
[0015] Preferably, the output result of the first recognition model is specifically: confirming the dialogue object of the speaker, taking the dialogue object as the target translation object, and outputting the translation language corresponding to the corresponding target translation object; if the dialogue object of the speaker cannot be confirmed, outputting each target translation language in turn.
[0016] Beneficial effect: The output result of the first recognition model can accurately determine the dialogue partner of the speaker, and when the dialogue partner cannot be determined, it can also output something to prevent the translation program from making mistakes. This can output one-to-one corresponding target translation languages according to different dialogue partners, with high flexibility.
[0017] Preferably, the personal keywords include identity titles, position titles, professional titles, occupational titles, names and personal pronouns.
[0018] Beneficial effect: By using the personal keywords in the speech information as input data, the speaker's dialogue partner can be accurately and quickly analyzed.
[0019] Preferably, the second acquisition module is also used to acquire surrounding environment features according to the visual sensor, and to acquire real-time application scenarios according to the surrounding environment features; the multi-language simultaneous translation intelligent robot also includes an environment judgment module, a translation extraction module and a second output module; the environment judgment module executes the translation extraction module if there are multiple speaking objects speaking at the same time; the translation extraction module is used to confirm that each speaking language corresponds to a different speaking object one by one, and then acquire the speech information of each speaking object; the second output module is used to take the speech information and the real-time application scenario as the second input data, and input the second input data into the second output model that has been trained; the second output model is used to output different key target translation information of each speaking object.
[0020] Beneficial effect: When there are at least multiple speakers speaking at the same time, the information extracted from different speakers is prone to mutual noise interference, which reduces the accuracy of recognition and translation. However, using the second input data and the corresponding second output model can improve the accuracy of the output.
[0021] Preferably, it also includes a reminder module, which is used to remind the target translation object in the same language as the key target translation information when outputting the key target translation information that the speech information of the speaking object cannot be fully translated.
[0022] Beneficial effect: The reminder module is designed to prevent output errors when the second output model is used to output, so as to prevent inaccurate information from being translated. Therefore, when the second output model is output, a prompt is given to avoid misunderstanding of the dialogue partner due to inaccurate output.
[0023] The second scheme: a multi-language simultaneous translation method, used to guide the multi-language simultaneous translation intelligent robot described in the first scheme, including: S1, obtaining the geographical location of the speaker; S2, obtaining the appearance attribute characteristics and audio characteristics of the speaker; S3, judging the speaking language of the speaker based on the geographical location of the speaker, the appearance attribute characteristics and audio characteristics of the speaker; S4, judging the target translation language required for the speaker to translate based on the audio characteristics of the speaker.
[0024] Beneficial effect: Compared with the existing technology, the race of the speaker is first determined by the appearance attributes of the speaker, and the speaking language of the speaker is determined based on the race, so as to achieve fast and accurate recognition without the speaker having to speak for a long time. Moreover, because it is in a complex environment where multiple languages coexist, it is not disturbed by the speaker's timbre and accurate speech information, which improves the recognition speed of the speaking language and reduces the difficulty of language recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 A multilingual simultaneous translation method for at least one embodiment;
[0026] Figure 2 A structural block diagram of a device according to at least one embodiment. DETAILED DESCRIPTION
[0027] The reference numerals in the drawings of the specification include:
[0028] Processor 101 , input device 102 , output device 103 , memory 104 , bus 105 , computer program 1041 .
[0029] Embodiment 1
[0030] This embodiment provides a multi-language simultaneous translation intelligent robot, which is equipped with a positioning sensor, a visual sensor and an auditory sensor. The intelligent robot includes a first acquisition module, which is used to obtain the geographical location characteristics of the speaking object according to the positioning sensor, and obtain the language existing in the geographical location according to the geographical location characteristics, and the existing languages include common languages and minority languages. Among them, common languages include English, French, Chinese, German, Spanish, Russian, Hindi, Turkic, Japanese, Korean, Arabic, Jewish, Australian, Brazilian, Korean and Mongolian. Minority languages include Tibetan, Urdu, Uyghur, Kazakh, Kirgiz, Xibe, Tajik, Kyrgyz, Iranian, Georgian and Uzbek. The robot is equipped with a memory 104, which is not only used to store the corresponding translations of each language, but also to classify the minority languages and common languages used in each region. And they are classified according to the race corresponding to their languages.
[0031] The second acquisition module is used to acquire the appearance attribute characteristics of the speaker according to the visual sensor, and acquire the language used by the speaker according to the appearance attribute characteristics. The appearance attribute characteristics include facial features, clothing, accessories and the eye gaze direction of the speaker. The clothing includes Hanfu, cheongsam, Tang suit, sari, Kurta-pagadi, Burqa, Hijab, Atlas Silk clothing, flower hat, dupatta, embroidered vest, kalpak or elechek (large headscarf worn by married women), Mongolian robe, high leather boots, long robe and embroidered shoes, rubat, kok, jiaguan and dopa.
[0032] The third acquisition module is used to acquire the audio features of the speaker according to the auditory sensor, and acquire the language used by the speaker according to the audio features.
[0033] Audio features include timbre, loudness, rhythm, emotional color, speech clarity and speech information; speech information includes key words and key phrases. Speech clarity refers to how accurately the vocal organs produce each phoneme. Clarity affects the difficulty of understanding speech.
[0034] The first judgment module judges whether the language obtained in the second acquisition module is within the range of languages obtained in the first acquisition module; if it is within the range of languages obtained in the first acquisition module, the language obtained in the second acquisition module is used as the speech language of the speaker; otherwise, the language obtained in the third acquisition module is used as the speech language of the speaker. The fourth acquisition module determines the speech language of the speaker and obtains the target translation language required for translation of the speaker according to the audio features.
[0035] Specifically, the appearance attribute features obtained by the visual sensor include facial features, clothing and accessories. The appearance attribute features are used as appearance attribute input data and input into the trained appearance recognition model. The appearance recognition model is used to output the ethnicity of the user according to the appearance attribute features of the user, and determine the language of the user according to the ethnicity. In this embodiment, the model can be an existing image recognition algorithm model, or a hybrid image recognition algorithm, or a deep learning model such as a convolutional neural network (CNN), a recurrent neural network (RNN) or a deep neural network (DNN). Regardless of whether an existing algorithm model is used, the algorithm model has been pre-created, trained and debugged by conventional technical means. The training data or input data of the image recognition algorithm model are generally clothing, accessories and facial features of ethnic minorities, so the output of the image recognition algorithm model generally includes two output results. The first output result is to recognize that the speaking object belongs to an ethnic minority, and then directly output the language corresponding to the ethnic minority. The second output result is that if the speaker is not identified as having ethnic minority characteristics, the third acquisition module is executed. When executing the third acquisition module, the audio features analyzed by the speaker are first used to determine whether the speech language belongs to the common language of the area. If it does not belong to the common language of the area, then determine whether the speech belongs to the ethnic minority language of the area; if neither of the first two belongs, then determine whether it conforms to the remaining languages. In this way, not only the recognition speed can be improved, but also the recognition accuracy rate can be improved, because the audio information may conform to multiple languages at the same time during the robot recognition process. In this embodiment, the audio features of the speaker are obtained by the auditory sensor, and then the audio features are analyzed according to the existing technical means to obtain the speech language of the speaker and the target translation language.
[0036] The beneficial effects of this embodiment
[0037] Different from the prior art, this embodiment uses two methods to determine the speaking language of the speaking object. First, the speaking object's race is determined based on the speaking object's appearance attributes, such as facial features, clothing and accessories, and then the speaking language corresponding to the race is determined to be consistent with the common language or minority language in the area. If it is consistent, the speaking language of the speaking object can be directly confirmed. The speaking language of the speaking object can be determined in a short time through the speaking object's appearance attributes. If it cannot be determined by the appearance attributes, the speaking language of the speaking object can be determined based on the audio features of the speaking object. In the prior art, it is believed that the speaking information of the speaking object is analyzed to analyze the speaking language of the speaking object. This is because the existing technical personnel believe that the speech content is the most direct, reliable and efficient basis for determining the language, and there is no necessary connection between appearance and language ability. In this embodiment, it is considered that there is an environment with complex and diverse languages. This environment is generally in an area where ethnic minorities gather. The facial features, clothing and costumes of different ethnic minorities are different, and there are large differences in the languages of ethnic minorities. There are also differences in the timbre of different ethnic minorities, which is not easy to identify. In this embodiment, the languages of the regions to which the speakers belong are summarized, and then the ethnicity of the speakers is determined based on the speakers' image characteristics to analyze the languages used by the speakers. In this way, in such a complex language environment, not only the recognition speed of the speakers' languages is improved, but also the speakers' languages can be accurately analyzed.
[0038] Embodiment 2
[0039] Different from the above-mentioned embodiment, this embodiment provides a multi-language simultaneous translation intelligent robot, and further includes: a multi-language translation quantity judgment module, which executes a target confirmation module if there are at least three speakers and at least three target translation languages at the same time; a target confirmation module, which corresponds each speech language to different speakers one by one according to the appearance attribute characteristics of the speaker and the audio characteristics of the speaker. A first recognition module, which obtains the eye sighting direction of the speaker according to the visual sensor, and uses the eye sighting direction as the first visual data. A second recognition module, which obtains the personal keywords of the speaker and the number of mentions of the personal keywords according to the auditory sensor, and uses the personal keywords and the number of mentions of the personal keywords as the first audio data. A translation language recognition module, which is used to use at least one of the first visual data and the first audio data as the first input data, and input the first input data into the first recognition model that has been trained; the first recognition model is used to output at least one target translation language to the target translation object. The output result of the first recognition model is specifically: confirming the dialogue object of the speaker, taking the dialogue object as the target translation object, and outputting the translation language corresponding to the corresponding target translation object; if the dialogue object of the speaker cannot be confirmed, then outputting each target translation language in turn. In this embodiment, the model can be an existing content recommendation algorithm model, or an existing filtering algorithm model based on collaborative recommendation, or a hybrid recommendation algorithm, or a deep learning model such as a convolutional neural network (CNN), a recurrent neural network (RNN) or a deep neural network (DNN). Regardless of whether an existing algorithm model is used, the algorithm model has been pre-created, trained and debugged by conventional technical means.
[0040] In this embodiment, if there are three speakers, A, B, and C, and the three speakers use different languages, three target translation languages are generated. By combining the visual sensor and the auditory sensor, the three speakers' facial features are detected to determine whether they change synchronously with the sound, thereby confirming the languages used by each of the three speakers. And by obtaining the eye gaze direction of the speaker who is speaking, the speaker's personal keywords, and the number of times the personal keywords are mentioned, it is determined to which dialogue partner the speaker's speech information is mainly directed. After confirmation, the target translation language required by the dialogue partner is translated to the dialogue partner.
[0041] Personal keywords include identity terms, job titles, professional terms, occupational terms, names, and personal pronouns. Job titles specifically refer to titles related to work or occupations. Professional terms refer to occupations with specific educational backgrounds or certifications. Occupational terms are mainly used to describe people who hold specific positions within an organization or institution.
[0042] Specifically, in this embodiment, there are three speakers, three different languages, and each speaker corresponds to a different language. The eye gaze direction of the speaker is used as the first visual data, which represents the direction of the speaker's eyes when speaking, whether it is facing the dialogue object. Further, whether it is facing a single dialogue object, if it is only facing a single dialogue object. The direction of the speaker's visual orientation is obtained through a visual sensor as an important visual data of the dialogue object of the speaker. Secondly, when extracting personal keywords from the speech information of the speaker, such as multiple mentions of personal pronouns such as "you", "he", "you", "we", and "they" in the sentence, it can be represented that the speaker's speech information specifically wants to have a dialogue with that dialogue object. The speech language of the speaker is used as the target translation language for other objects.
[0043] The beneficial effects of this embodiment
[0044] Compared with the prior art, only two-way translation is performed. When there are at least three speakers using different languages in a discussion, direct and complete word-for-word translation is performed, and language switching and inaccurate recognition are prone to occur.
[0045] This embodiment is mainly aimed at accurately identifying the speech language corresponding to each speaker when at least three speakers in different languages are communicating and discussing, and judging whether the speech of the speaker is directed to an individual or the entire group based on the speech information of the speaker, and judging the dialogue object to which the speech information is mainly conveyed based on the speech information of the speaker. In this way, only accurate and one-to-one translation is required to allow the dialogue object to understand the content expressed by the speaker. In this way, not only the number of translations into all languages is reduced, the amount of data calculation is reduced, but also the speech information of the speaker can be accurately translated to the dialogue object, the output speed of the translated language is improved, and the output method of the translated language is changed.
[0046] Embodiment 3
[0047] Different from the above-mentioned embodiment, the present embodiment provides a multi-language synchronous translation intelligent robot, and further includes: a second acquisition module, the second acquisition module is also used to acquire the surrounding environment characteristics according to the visual sensor, and acquire the real-time application scenario according to the surrounding environment characteristics; the multi-language synchronous translation intelligent robot also includes an environment judgment module, a translation extraction module and a second output module; the environment judgment module, if there are multiple speaking objects speaking at the same time, executes the translation extraction module; the translation extraction module is used to confirm that each speech language corresponds to a different speaking object one by one, and then acquire the speech information of each speaking object; the second output module is used to take the speech information and the real-time application scenario as the second input data, and input the second input data into the second output model that has been trained; the second output model is used to output the key target translation information. It also includes a reminder module, which is used to remind the target translation object in the same language as the key target translation information when outputting the key target translation information, and the speech information of the speaking object cannot be fully translated. In this embodiment, the model can be an existing content recommendation algorithm model, an existing collaborative recommendation-based filtering algorithm model, or a hybrid recommendation algorithm, or a deep learning model such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a deep neural network (DNN). Regardless of whether an existing algorithm model is used, the algorithm model has been pre-created, trained, and debugged by conventional technical means.
[0048] Specifically, when there are multiple speaking objects speaking at the same time, the robot extracts the speech information of each speaking object at this time, which can be extracted according to the different audio features of different speaking objects, such as judging from the rhythm, emotional color, voice clarity and language. The target confirmation module can be used to refer to each speech language and correspond each speech object to different speaking objects. At this time, multiple speakers speak at the same time, and when extracting the speech information of the speaking object, many speech information is incomplete. If it is a robot of the prior art, all the recognized speech information will be merged and output, which is equivalent to translating after combining the speech information of all speaking objects. This will cause great interference and very low accuracy. However, this embodiment is different. This embodiment first extracts the speech information of different speaking objects, and then outputs different translation information according to the speech information of different speaking objects. In order to further increase the accuracy of the translation, by using the real-time application scenario as the second input data, it is equivalent to storing the corresponding information in the real-time application scenario in the memory 104. The application scenarios include museums, academic conferences, shops, government and enterprise service halls, and law enforcement venues. These application scenarios will include commonly used related information, and these related information are also used to train the second output model. The speech information of different speakers is output through the second output model, and the speech information of the speakers can also be polished and output, further improving the accuracy of the translation.
[0049] In another embodiment, the probability of occurrence of the speech information can be compared with the information of the application scenario when extracting the speech information of the speech object:
[0050]
[0051] In formula (1), λ(α+1) represents word alignment, which represents the conditional probability of the occurrence of related words in the speech information of the speaker in the application scenario. Formula (1) is calculated by calculating the conditional probability of the occurrence of the speech information in the application scenario, and formula (1) is added to the second output model to further calculate the similarity between the speech information and the application scenario, so as to further optimize the second output model.
[0052] The beneficial effects of this embodiment
[0053] When multiple speakers are discussing and communicating, it is easy for them to generate noise in each other's speech information, which can easily interfere with the translation of intelligent robots. In existing robot translation, accurate speech information must be obtained before translation can be performed, and when complete speech information cannot be obtained, existing robots are prone to delays in recognition and output, and it takes more time to extract information in different languages. Alternatively, existing robots only translate speech information when speaking simultaneously as one language, which greatly reduces the accuracy of translation.
[0054] However, in this embodiment, the robot's own visual sensor determines the real-time application scenario of the user, and the application scenario can be obtained through its own visual sensor, and polishing is performed based on some key information of the real-time application scenario and some speech information extracted by the speaking object, so as to output relatively accurate translated speech information in a short time. Not only can it translate the target translation language corresponding to each speaking object when multiple speaking objects speak at the same time, accurately determine the speech information of the speaking object, but also further polish the speech information of the speaking object, reduce the noise interference of other information, and at the same time, it can be further polished only based on the application scenario information, and the processing time is short.
[0055] Embodiment 4
[0056] The present embodiment provides a multi-language simultaneous translation method for guiding the multi-language simultaneous translation robot in the first embodiment, including S1, obtaining the geographical location of the speaker; S2, obtaining the appearance attribute characteristics and audio characteristics of the speaker; S3, judging the speaking language of the speaker according to the geographical location of the speaker, the appearance attribute characteristics and audio characteristics of the speaker; S4, judging the target translation language required for the speaker to translate according to the audio characteristics of the speaker.
[0057] Embodiment 5
[0058] like Figure 2 The device is used to be installed on a multilingual simultaneous translation intelligent robot. The installation method is a conventional method of technicians in this field. The device may include: one or more processors 101, one or more input devices 102, one or more output devices 103 and a memory 104. The processors 101, input devices 102, output devices 103 and memory 104 are interconnected via a bus 105. The memory 104 is used to store a computer program, which includes program instructions. The processor 101 is configured to call the program instructions to execute the method steps described in the above embodiments.
[0059] It should be understood that in the embodiment of the present invention, the processor 101 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0060] The input device 102 may include a keyboard, etc., and the output device 103 may include a display (LCD, etc.), a speaker, etc.
[0061] The memory 104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include a non-volatile random access memory. For example, the memory 104 may also store information on the device type.
[0062] In a specific implementation, the processor 101, input device 102, and output device 103 described in the embodiment of the present invention can execute the implementation method described in the embodiment of the multi-language simultaneous translation method provided by the embodiment of the present invention, which will not be repeated here.
[0063] It should be noted that for a more specific description of the working process of the device, please refer to the aforementioned method embodiment part, which will not be repeated here.
[0064] Embodiment 6
[0065] This embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, a multi-language simultaneous translation method is implemented.
[0066] The computer-readable storage medium may be the internal storage unit of the device described in the foregoing embodiments, such as a hard disk or memory of the system. The computer-readable storage medium may also be an external storage device of the system, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the system. The computer-readable storage medium is used to store other programs and data required by the computer program. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
[0067] The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.
[0068] In this specification, a lot of specific details are described. However, it is understood that embodiments of the present invention can be put into practice without these specific details. In some examples, known methods, systems and techniques are not shown in detail, so as not to obscure the understanding of this specification. In the description of this specification, the description of reference terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, methods, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification.
[0069] In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, systems, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.
[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.
Claims
1. Multi-language simultaneous translation intelligent robot, characterized by: Used to translate the language of the speaking object and pass it to the translation object, including: A first acquisition module is used to acquire geographical location features of a speaker according to a positioning sensor, and acquire the languages existing in the geographical location according to the geographical location features, wherein the existing languages include common languages and minority languages; A second acquisition module is used to acquire appearance attribute features of the speaker according to the visual sensor, determine the race of the speaker according to the appearance attribute features, and determine the language used by the speaker according to the race of the speaker; A third acquisition module is used to acquire the audio features of the speaker according to the auditory sensor, and acquire the language used by the speaker according to the audio features; The first judgment module judges whether the language obtained in the second acquisition module is within the range of languages obtained in the first acquisition module; if it is within the range of languages obtained in the first acquisition module, the language obtained in the second acquisition module is used as the speech language of the speech object; otherwise, the language obtained in the third acquisition module is used as the speech language of the speech object; The fourth acquisition module determines the speaking language of the speaker and acquires the target translation language required for translation of the speaker according to the audio features.
2. The multilingual simultaneous translation intelligent robot according to claim 1, characterized in that: The minority languages include Tibetan, Urdu, Uyghur, Kazakh, Kirgiz, Xibe, Tajik, Kyrgyz, Iranian, Georgian and Uzbek.
3. The multilingual simultaneous translation intelligent robot according to claim 1, characterized in that: The appearance attribute features include facial features, clothing, accessories and the eye gaze direction of the speaker.
4. The multilingual simultaneous translation intelligent robot according to claim 1, characterized in that: Audio features include timbre, loudness, rhythm, emotional color, speech clarity and speech information; among which, speech information includes key words and key phrases.
5. The multilingual simultaneous translation intelligent robot according to claim 3, characterized in that: Also includes: The language translation quantity judgment module executes the target confirmation module if there are at least three speakers and at least three target translation languages; A target confirmation module, which matches each speech language to a different speech object according to the appearance attribute characteristics of the speech object and the audio characteristics of the speech object; A first recognition module, which obtains the eye gaze direction of the speaker according to the visual sensor, and uses the eye gaze direction as the first visual data; A second recognition module, which obtains, according to the auditory sensor, a person keyword and a number of times the person keyword is mentioned in the speech information of the speaker, and uses the person keyword and the number of times the person keyword is mentioned as the first audio data; The translation language recognition module is used to take at least one of the first visual data and the first audio data as first input data, and input the first input data into a first recognition model that has been trained; the first recognition model is used to output at least one target translation language to multiple target translation objects.
6. The multilingual simultaneous translation intelligent robot according to claim 5, characterized in that: The output result of the first recognition model is specifically: confirming the dialogue object of the speaker, taking the dialogue object as the target translation object, and outputting the translation language corresponding to the target translation object; If the speaker's dialogue partner cannot be confirmed, each target translation language is output in sequence.
7. The multilingual simultaneous translation intelligent robot according to claim 5, characterized in that: The personal keywords include identity titles, position titles, professional titles, occupational titles, names and personal pronouns.
8. The multilingual simultaneous translation intelligent robot according to claim 4, characterized in that: The second acquisition module is also used to acquire surrounding environment features according to the visual sensor, and acquire the real-time application scene according to the surrounding environment features; The multi-language simultaneous translation intelligent robot also includes an environment judgment module, a translation extraction module and a second output module; The environment judgment module executes the translation extraction module if there are multiple speakers speaking at the same time; A translation extraction module, configured to obtain speech information of each speech object after confirming that each speech language corresponds to a different speech object one by one; A second output module, used to take the speech information and the real-time application scenario as second input data, and input the second input data into a trained second output model; The second output model is used to output different key target translation information for each speaker.
9. The multilingual simultaneous translation intelligent robot according to claim 8, characterized in that: The system also includes a reminder module, which is used to remind the target translation object in the same language as the key target translation information when outputting the key target translation information, and that the speech information of the speaking object cannot be completely translated.
10. A multilingual simultaneous translation method, characterized in that: The intelligent robot for guiding the multilingual simultaneous translation described in any one of claims 1 to 9 comprises: S1, obtaining the geographic location of the speaker; S2, obtaining appearance attribute features and audio features of the speaker; S3, determining the speaking language of the speaker according to the geographic location of the speaker, the appearance attribute characteristics and the audio characteristics of the speaker; S4, determining the target language that the speaker needs to translate into according to the audio features of the speaker.