Context translation processing method and device based on large model, and intelligent equipment

Abstract

The invention discloses a context translation processing method and device based on a large model, and intelligent equipment, and the method comprises the steps: obtaining voice information needing to be translated, and obtaining scene information corresponding to the voice information; inputting the processed voice information and the corresponding scene information into a pre-trained scene-based translation model, and outputting translation content corresponding to the scene; wherein the pre-trained training step based on the scene translation model comprises the following steps: collecting and acquiring different videos as training data; analyzing a frame-by-frame page, obtaining a subtitle file, and understanding a plot scene of each video according to video plot description and comment content; and based on the plot scene of each video, in combination with the frame-by-frame page of the video and the corresponding subtitle file, analyzing the corresponding translation content, and training the scene translation model to obtain a pre-trained scene-based translation model. The translation of the method is not only simple character translation, but also can be performed according to the scene in combination with the corresponding context, so that the translation is more accurate, and the translation effect is better.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a contextual translation processing method, apparatus, intelligent device and storage medium based on a large model. Background Art

[0002] With the development of science and technology and the continuous improvement of people's living standards, the use of various smart terminals is becoming more and more popular, and many smart terminals have translation functions.

[0003] However, the translation of the smart terminal in the prior art is just a simple text translation, which is basically unable to achieve translation according to the scene and the corresponding context, and the translation effect needs to be improved.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the problems and defects of the above-mentioned prior art, a contextual translation processing method, apparatus, intelligent device and storage medium based on a large model are provided. The translation of the present invention is not just a simple text translation, but can also be translated according to the scene combined with the corresponding context, so that the translation is more accurate and the translation effect is better.

[0006] The technical solution adopted by the present invention to solve the problem is as follows:

[0007] A contextual translation processing method based on a large model, comprising:

[0008] When the translation function is detected to be turned on, the language of the associated translation is controlled, and the voice information to be translated is obtained, as well as the scene information corresponding to the voice information is obtained;

[0009] Input the processed speech information and the corresponding scene information into a pre-trained scene-based translation model, and output the translation content corresponding to the scene;

[0010] The training steps of the pre-trained scene-based translation model include:

[0011] Collect different videos as training data;

[0012] Analyze the frame-by-frame pages of the video to obtain corresponding subtitle files;

[0013] and obtaining and analyzing the video plot descriptions and comments, and understanding the plot scenes of each video;

[0014] Based on the plot scenes of each video, combined with the video frame-by-frame pages and corresponding subtitle files, the corresponding translation content is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model.

[0015] In the contextual translation processing method based on a large model, when the translation function is detected to be turned on, the step of controlling the language of the associated translation includes:

[0016] Determining whether a translation request instruction is detected;

[0017] When a translation request instruction is detected, the control starts the translation function and provides an interface for selecting a target language for translation;

[0018] Receive user's operation instructions to confirm the target language to be translated, and control the associated translation language.

[0019] In the contextual translation processing method based on a large model, the steps of obtaining the voice information to be translated and obtaining the scene information corresponding to the voice information include:

[0020] Based on the determined target language of translation, the voice information to be translated is extracted from the video file or voice file to be translated;

[0021] The voice information that needs to be translated is identified and converted into translatable text content through voice signal processing and natural language processing technology;

[0022] According to the geographical location scene, conversation content and context information of the video file or voice file to be translated, the specific scene of the video file or voice file to be translated is obtained.

[0023] In the contextual translation processing method based on a large model, the geographical location scene is the actual location where the video or voice content occurs;

[0024] The dialogue content includes the subject, emotion, and role relationship involved in the dialogue;

[0025] The context information includes the conversation content and the preceding and following context information.

[0026] The contextual translation processing method based on a large model, wherein the step of inputting the processed speech information and the corresponding scene information into a pre-trained scene-based translation model and outputting the translation content corresponding to the scene comprises:

[0027] The speech information to be translated is converted into text through speech recognition, and the scene information corresponding to the speech information is used as the input of the model and input into the pre-trained scene-based translation model;

[0028] The pre-trained scenario-based translation model analyzes the original text according to the input text and scenario information, and generates a translation result in combination with the tone, wording and grammatical structure of the scenario.

[0029] In the contextual translation processing method based on a large model, the steps of obtaining the video plot description and comment content, analyzing the video plot description and comment content, and understanding the plot scenes of each video include:

[0030] Collect a plot summary of the film's overall plot, as well as review content from audience or critics;

[0031] Understand the context of the video and conduct in-depth analysis of the plot, characters, and possible emotional tone based on the collected plot summary description of the overall plot of the movie and the comments fed back by the audience or critics.

[0032] In the contextual translation processing method based on a large model, the training step of the pre-trained scene-based translation model further includes:

[0033] Collect and download different movie videos as training data;

[0034] Analyze the frame-by-frame pages of the video and obtain the corresponding movie subtitle files;

[0035] Search movies online on Douban interface, obtain movie plot descriptions and user reviews, analyze plot descriptions and user reviews, and understand plot scenes;

[0036] Based on the plot scenes of each video, combined with the frame-by-frame pages and subtitle files, the translation content of each language is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model.

[0037] A contextual translation processing device based on a large model, wherein the device comprises:

[0038] The pre-training module is used to collect and obtain different videos as training data; analyze the frame-by-frame pages of the videos to obtain corresponding subtitle files; and obtain the video plot descriptions and commentary content, analyze the video plot descriptions and commentary content, and understand the plot scenes of each video; based on the plot scenes of each video, combined with the video frame-by-frame pages and corresponding subtitle files, analyze the corresponding translation content, train the scene translation model, and obtain a pre-trained scene-based translation model.

[0039] A translation acquisition module, which is used to control the language of the associated translation when detecting that the translation function is turned on, and to obtain the voice information to be translated, and to obtain the scene information corresponding to the voice information;

[0040] The translation application module is used to input the processed voice information and the corresponding scene information into a pre-trained scene-based translation model and output the translation content corresponding to the scene.

[0041] An intelligent device includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the one or more programs including the method for executing any one of the methods described above.

[0042] A computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the methods described above.

[0043] Beneficial effects of the present invention: The present invention provides a contextual translation processing method, device, intelligent device and storage medium based on a large model. When the translation function is detected, the language of the associated translation is controlled, and the voice information to be translated is obtained, and the scene information corresponding to the voice information is obtained; the processed voice information and the corresponding scene information are input into a pre-trained scene-based translation model, and the translation content corresponding to the scene is output; wherein the training steps of the pre-trained scene-based translation model include: collecting and obtaining different videos as training data; analyzing the frame-by-frame pages of the video to obtain the corresponding subtitle files; and obtaining the description and comment content of the video plot, analyzing the description and comment content of the video plot, and understanding the plot scenes of each video; based on the plot scenes of each video, combined with the frame-by-frame pages of the video and the corresponding subtitle files, the corresponding translation content is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model. The translation of the present invention is not just a simple text translation, but also can be translated according to the scene combined with the corresponding context, so that the translation is more accurate and the translation effect is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a flowchart of a contextual translation processing method based on a large model provided in an embodiment of the present invention.

[0046] Figure 2 It is a flowchart of the contextual translation processing method based on a large model provided in Example 2 of the present invention.

[0047] Figure 3 It is a principle block diagram of an embodiment of a contextual translation processing device based on a large model provided by the present invention.

[0048] Figure 4 It is a block diagram of the internal structure principle of the smart device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0051] The translation of smart terminals in the prior art is simply text translation, and it is basically impossible to translate according to the scene and the corresponding context, and the translation effect needs to be improved.

[0052] The present invention proposes a contextual translation processing method based on a large model. The translation of the present invention is not just a simple text translation, but can also achieve translation according to the scene combined with the corresponding context, so that the translation is more accurate and the translation effect is better.

[0053] like Figure 1 As shown, a contextual translation processing method based on a large model according to Embodiment 1 of the present invention comprises the following steps:

[0054] Step S100: When it is detected that the translation function is turned on, the language associated with the translation is controlled, and the voice information to be translated is obtained, and the scene information corresponding to the voice information is obtained;

[0055] In the embodiment of the present invention, after the translation function is turned on, it is necessary to control multiple languages ​​and obtain relevant voice information and scene information.

[0056] Specifically, when a user requests translation, the present invention can monitor the user's instructions through voice recognition technology, or monitor manual operation instructions, to confirm that the user wants to use the translation function. After confirming the user's intention, the present invention will perform subsequent operations.

[0057] In the embodiment of the present invention, an interface for selecting a language is provided. For example, the user can choose to translate into English, Spanish, or Chinese, etc. This can not only ensure the accuracy of the translation, but also adjust the translation language in real time according to the user's needs.

[0058] In the embodiment of the present invention, once the translation function is turned on and the target language of the translation is determined, the text information to be translated will be extracted from the material to be translated (such as a video file or a voice file). This involves the technology of speech signal processing and natural language processing to recognize and convert speech into translatable text.

[0059] In the embodiment of the present invention, in addition to obtaining the text content to be translated, effective translation also requires understanding the context. The present invention will determine the specific scene of the speech segment, such as a business meeting, a travel guide, or ordinary chat, based on the geographical location scene, conversation content, and other contextual information of the material to be translated (such as a video file or a voice file). This scene information helps to decide on more accurate words and expressions in the translation process to ensure natural and smooth translation.

[0060] For example, when a user needs to translate his speech into Spanish at a business meeting, first, the present invention can detect that the user has turned on the translation function. Next, the user selects Spanish as the target language. Then, the user starts speaking, and the present invention can obtain "our market share is growing year by year" through voice recognition.

[0061] At the same time, the present invention also analyzes the usage scenario of the translation, and the extracted scenario information will affect the subsequent translation results. For example, in a business environment, the present invention can translate "market share" into "cuota de mercado" instead of daily words, so that the translated sentence will be more formal and appropriate.

[0062] The advantage of the present invention is that by controlling the language and obtaining the scene information, a more accurate translation can be provided, so that the user can obtain a translation result that is more in line with the actual needs. And understanding the scene background of the user can make the translation result more targeted, avoid unnecessary misunderstandings and ambiguous expressions, and thus improve the user experience.

[0063] It can be seen that the present invention and this process not only improve the efficiency and accuracy of translation, but also provide users with a higher quality service experience.

[0064] Specifically, the step S100 includes:

[0065] S101, based on the determined target language for translation, extracting voice information to be translated from a video file or a voice file to be translated;

[0066] In the embodiment of this step, when the target language to be translated is selected, such as English into Chinese, the voice information to be translated is extracted from the video file or voice file to be translated, such as separating the English audio from the video file or voice file to be translated.

[0067] S102, identifying and converting the voice information to be translated into translatable text content through voice signal processing and natural language processing technology;

[0068] In this step, the voice information to be translated is identified and converted into translatable text content through voice signal processing and natural language processing technology. For example, English audio is converted into text to obtain English lines.

[0069] S103, obtaining a specific scene of the video file or voice file to be translated according to the geographical location scene, conversation content and context information of the video file or voice file to be translated;

[0070] In this step, before translating the video or audio file, the geographical location, conversation content and context information will be analyzed in detail to clarify the specific scene. The purpose of this process is to ensure accuracy and appropriateness in the translation process, because different cultural and environmental backgrounds may affect the understanding and expression of the conversation.

[0071] The geographic location scene refers to the actual location where the video or voice content takes place, such as a busy city street, a village, a school, or an office. This scene affects the wording and tone of the conversation. For example, in a formal business meeting, the conversation may be more professional and formal, while a casual chat among friends may be more casual.

[0072] Regarding the content of the conversation, the present invention analyzes the topics, emotions, role relationships, etc. involved in the conversation. For example, if the conversation is about wedding arrangements, the translation may need to pay special attention to cultural customs related to weddings, which may vary from place to place.

[0073] About context information: It includes not only the content of the conversation, but also the information of the context before and after, such as who are the people involved in the conversation, what is the relationship between them, what topics have been discussed before, etc. This information can help translators better understand the intention and make more appropriate translations.

[0074] For example, in the present invention, there is a video about "Spring Festival Celebrations", in which a group of people prepare New Year's Eve dinner at home. If the translation is only performed from the language level, focusing only on the conversion of words, the cultural connotation and customs of the Spring Festival may be omitted.

[0075] Then, in the present invention, regarding the geographical location scenario, if the present invention knows that the video was shot inside the Chinese community, the translator can more easily understand the importance of "reunion dinner" and emphasize the theme of "family reunion" when translating.

[0076] Regarding the content of the dialogue, in the present invention, if a specific food such as "dumplings" or "rice cakes" is mentioned in the video, the translator may consider adding an explanation in the translation, such as "rice cakes symbolize higher achievements in the coming year."

[0077] Regarding contextual information, in the present invention, if a previous family celebration is mentioned in the conversation, this level of information can be added during translation to make the translation more complete and appropriate.

[0078] It can be seen that in this way, translators are able to provide not only linguistic translation, but also cultural communication, making the target audience more resonant and relevant when understanding the content.

[0079] By understanding the specific scenario, the present invention can select more accurate words and expressions for translation, avoiding misunderstandings caused by cultural differences. In addition, the geographical and cultural background is integrated into the translation, making the dialogue more in line with the expression habits of the target language and sounding more natural.

[0080] Step S200: input the processed voice information and the corresponding scene information into a pre-trained scene-based translation model, and output the translation content corresponding to the scene;

[0081] In the embodiment of the present invention, the extracted voice information and scene information are input into a pre-trained translation model to generate translation content that matches the specific scene.

[0082] Specifically, the speech information to be translated is converted into text through speech recognition as the input of the model. This string of text contains the content that the user wants to translate.

[0083] And combined with scene information, that is, in the embodiment of the present invention, in addition to the voice information to be translated, the acquired scene information is also input. This scene information may include the context of the content to be translated, the use occasion (such as business, travel, social, etc.), the geographical location, etc. This part of information is the key to contextualizing the text content.

[0084] The present invention will call a pre-trained translation model, that is, the system will simultaneously input the above information into a scene translation model that has been trained with a large amount of data. This model has learned the language usage habits and professional terms in different contexts, and can generate appropriate translations based on the input content and scene information.

[0085] Then, according to the input text and scene information, the model will generate a translation result. This result should not only accurately translate the meaning of the original text, but also conform to the tone, wording and grammatical structure of the scene.

[0086] For example, imagine a scenario where a doctor is communicating with a foreign patient, and the patient describes their symptoms in English (such as "I feel like my headache has lasted for three days."). The doctor uses the translation processing method of the present invention to turn on the translation function. The voice information, i.e., the user's symptom description, is first extracted.

[0087] At the same time, the present invention also obtains the scene information of "medical environment", indicating that the conversation is an exchange between a professional doctor and a patient.

[0088] When this information is input into the pre-trained scenario-based translation model, the pre-trained scenario-based translation model of the present invention will take into account the professional terms and grammatical structures in medical situations and process them accordingly.

[0089] The final output translation might be: "He estado sintiendo un dolor de cabeza queha durado tres días." This Spanish not only accurately conveys the symptoms, but also uses appropriate expressions in a medical context.

[0090] It can be seen that in this way, the present invention not only retains the meaning of the original text in the translation, but also fully considers the scene context, thereby improving the practicality and effectiveness of the translation. This is the significance of combining voice information with scene information and using a pre-trained model for translation.

[0091] The benefit of the present invention is that by taking context information into consideration, the translation content can better adapt to the specific environment and improve the effectiveness of its practical application. For example, in professional fields such as law and medicine, the use of professional terms will enhance the authority and accuracy of the translation.

[0092] It can also improve communication efficiency because the translated content will be adjusted according to the scenario, allowing users to communicate more fluently and smoothly in specific situations, reducing the possibility of misunderstanding or confusion, thereby improving communication efficiency.

[0093] It can also improve user satisfaction because users can get translation results that meet their needs. This customized translation service can greatly improve user satisfaction and increase user trust in the system.

[0094] In the embodiment of the present invention, the training step of the pre-trained scene-based translation model includes:

[0095] S201: Collect and obtain different videos as training data;

[0096] The goal of this step is to collect a wide range of diverse video materials. These videos should cover different topics, styles, and cultural backgrounds to ensure the diversity of training data. The collected videos should include various types of dialogues (such as formal, informal, humorous, etc.), as well as works from different regions or areas.

[0097] S202: Analyze the frame-by-frame pages of the video to obtain corresponding subtitle files;

[0098] In this step, the video is analyzed frame by frame to gain a deep understanding of the content displayed in each frame, including visual and language information. Through this process, subtitle files related to the video content can be extracted. These subtitle files not only provide a text version of the dialogue, but also provide the model with a semantic understanding of the timeline.

[0099] S203: Obtaining a video plot description and comment content, and analyzing the video plot description and comment content;

[0100] In this step, the collected video plot descriptions and comments are analyzed. These texts are usually divided into two categories: one is a summary of the overall plot of the movie, and the other is the feedback from the audience or critics. These descriptions and comments can help understand the context of the video and deeply analyze the plot, characters, and possible emotional tone. It is convenient for the model to capture deeper semantics.

[0101] S204: Based on the plot scenes of each video, combined with the video frame-by-frame pages and corresponding subtitle files, the corresponding translation content is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model;

[0102] In this step, based on the data collected and analyzed above, the translation model is trained in combination with the frame-by-frame information of the video, the subtitle file, and the plot description. In this process, deep learning algorithms can be used to analyze the language features in different scenarios and optimize the translation results. The final model will be able to translate dialogues in specific scenarios more accurately.

[0103] For example, in the embodiment of the present invention, when you want to build a translation model for comedy movies, you can implement it through the following implementation steps:

[0104] First collect video (S201): For example, collect a series of popular comedy films, including "A Chinese Odyssey", "Beautiful Life", etc. These films have humorous and exaggerated dialogue styles.

[0105] Then analyze frame by frame and subtitles (S202): In this embodiment, "Da Hua X You" can be analyzed frame by frame to extract corresponding subtitles to understand the dialogue and body language of the characters in each important scene.

[0106] Obtaining plot descriptions and comments (S203): Obtaining audio-visual comments and plot descriptions from major review websites on the Internet, such as the audience's views on the emotional entanglement between "Supreme Treasure and Fairy Zixia".

[0107] Then, model training (S204) is performed: specifically, the model can be trained by combining frame-by-frame information and subtitles to make it understand the changes in tone between "joke" and "sarcasm", so that the comedy effect can be retained and humorous elements can be presented during translation.

[0108] Through frame-by-frame analysis and plot understanding, the model in this embodiment can grasp the context and generate a translation that is more in line with the scene, avoiding the errors of mechanical translation. In addition, the training data of different types of videos makes the model more universally applicable and can meet the translation needs from different cultural and language backgrounds. In addition, combined with the comment content and plot analysis, the model can understand the emotional expression in different scenes, thereby improving the emotional accuracy and delicacy of the translation.

[0109] It can be seen that in this specific embodiment, through such a process, a more accurate translation model that conforms to a specific scenario can be obtained, which not only improves the quality of the translation, but also makes the translation closer to the original meaning and emotional level of the film.

[0110] The present invention is further described in detail below through specific application examples;

[0111] This specific application embodiment uses the context translation processing method based on a large model to be applied to a video translation player as an example for explanation:

[0112] like Figure 2 As shown, the context translation processing method based on the large model provided in the second embodiment includes the following steps:

[0113] Step S10, start the video translation player development, and proceed to step S11;

[0114] Step S11, collect training data, and proceed to step S12;

[0115] Step S12, download different movie videos, and then proceed to step S13;

[0116] In the present invention, different movie videos are collected and downloaded as training data;

[0117] Step S13, providing a video frame-by-frame page, and proceeding to step S14;

[0118] Step S14, download the corresponding movie subtitle file, and proceed to step S15;

[0119] The present invention analyzes the frame-by-frame pages of the video and obtains the corresponding movie subtitle files;

[0120] Step S15, searching for movies online on the corresponding Douban interface, and proceeding to step S16;

[0121] Step S16, obtain movie plot description and user comments, and then proceed to step S17.

[0122] Step S17, analyzing the plot description and user comments, and proceeding to step S18;

[0123] Step S18, understand the plot, and proceed to step S19;

[0124] In the embodiment of the present invention, the Douban interface corresponding to the movie is searched online, the movie plot description and user comments are obtained, the plot description and user comments are analyzed, and the plot scenes are understood;

[0125] Step S19, combining the frame-by-frame page and the subtitle file, and proceeding to step S20;

[0126] Step S20, analyzing the Chinese-English translation content, and proceeding to step S21;

[0127] Step S21, train the large model to understand the context, and proceed to step S22;

[0128] In this specific embodiment, based on the plot scenes of each video, combined with the frame-by-frame pages and subtitle files, the translation content of each language is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model;

[0129] Step S22, applying the large model to video translation, and proceeding to step S23;

[0130] Step S23, generate accurate subtitles, and proceed to step S24;

[0131] Step S24: providing translated video content.

[0132] It can be seen that in the embodiment of the present invention, the existing subtitle files on the Internet are bound to Douban's introduction and comments on the movie, and then combined with frame-by-frame picture analysis, so that the context can be automatically built and trained. Based on the plot scenes of each video, combined with frame-by-frame pages and subtitle files, the translation content of each language is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model. Then, using the pre-trained scene-based translation model for translation, a scene-based translation result can be obtained, making the translation more accurate.

[0133] Exemplary Devices

[0134] like Figure 3 As shown, an embodiment of the present invention provides a context translation processing device based on a large model, the device comprising:

[0135] The pre-training module 310 is used to collect and obtain different videos as training data; analyze the frame-by-frame pages of the videos to obtain corresponding subtitle files; and obtain the video plot descriptions and commentary content, analyze the video plot descriptions and commentary content, and understand the plot scenes of each video; based on the plot scenes of each video, combined with the video frame-by-frame pages and corresponding subtitle files, analyze the corresponding translation content, train the scene translation model, and obtain a pre-trained scene-based translation model.

[0136] The translation acquisition module 320 is used to control the language of the associated translation when detecting that the translation function is turned on, and to obtain the voice information to be translated, and to obtain the scene information corresponding to the voice information;

[0137] The translation application module 330 is used to input the processed voice information and the corresponding scene information into a pre-trained scene-based translation model, and output the translation content corresponding to the scene, as described above.

[0138] Based on the above embodiments, the present invention further provides an intelligent device, whose principle block diagram can be shown as follows: Figure 4 As shown. The smart device includes a processor, a memory, a network interface, a display screen, and a database connected through a system bus. Among them, the processor of the smart device is used to provide computing and control capabilities. The memory of the smart device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the smart device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a context translation processing method based on a large model is implemented. The database of the smart device is used to store a context translation processing program based on a large model.

[0139] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the smart device to which the solution of the present invention is applied. The specific smart device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, a smart device is provided, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing the following operations:

[0141] When the translation function is detected to be turned on, the language of the associated translation is controlled, and the voice information to be translated is obtained, as well as the scene information corresponding to the voice information is obtained;

[0142] Input the processed speech information and the corresponding scene information into a pre-trained scene-based translation model, and output the translation content corresponding to the scene;

[0143] The training steps of the pre-trained scene-based translation model include:

[0144] Collect different videos as training data;

[0145] Analyze the frame-by-frame pages of the video to obtain corresponding subtitle files;

[0146] and obtaining and analyzing the video plot descriptions and comments, and understanding the plot scenes of each video;

[0147] Based on the plot scenes of each video, combined with the video frame-by-frame pages and corresponding subtitle files, the corresponding translation content is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model; the details are as described above.

[0148] Wherein, when the translation function is detected to be turned on, the step of controlling the language associated with the translation includes:

[0149] Determining whether a translation request instruction is detected;

[0150] When a translation request instruction is detected, the control starts the translation function and provides an interface for selecting a target language for translation;

[0151] Receive user's operation instructions to confirm the target language to be translated, and control the associated translation language.

[0152] The steps of obtaining the voice information to be translated and obtaining the scene information corresponding to the voice information include:

[0153] Based on the determined target language of translation, the voice information to be translated is extracted from the video file or voice file to be translated;

[0154] The voice information that needs to be translated is identified and converted into translatable text content through voice signal processing and natural language processing technology;

[0155] According to the geographical location scene, conversation content and context information of the video file or voice file to be translated, the specific scene of the video file or voice file to be translated is obtained.

[0156] The geographical location scene is the actual location where the video or voice content occurs;

[0157] The dialogue content includes the subject, emotion, and role relationship involved in the dialogue;

[0158] The context information includes the conversation content and the preceding and following context information.

[0159] The step of inputting the processed voice information and the corresponding scene information into a pre-trained scene-based translation model and outputting the translation content corresponding to the scene includes:

[0160] The speech information to be translated is converted into text through speech recognition, and the scene information corresponding to the speech information is used as the input of the model and input into the pre-trained scene-based translation model;

[0161] The pre-trained scenario-based translation model analyzes the original text according to the input text and scenario information, and generates a translation result in combination with the tone, wording and grammatical structure of the scenario.

[0162] The steps of obtaining the video plot description and comment content, analyzing the video plot description and comment content, and understanding the plot scenes of each video include:

[0163] Collect a plot summary of the film's overall plot, as well as review content from audience or critics;

[0164] Understand the context of the video and conduct in-depth analysis of the plot, characters, and possible emotional tone based on the collected plot summary description of the overall plot of the movie and the comments fed back by the audience or critics.

[0165] The training step of the pre-trained scene-based translation model further includes:

[0166] Collect and download different movie videos as training data;

[0167] Analyze the frame-by-frame pages of the video and obtain the corresponding movie subtitle files;

[0168] Search movies online on Douban interface, obtain movie plot descriptions and user reviews, analyze plot descriptions and user reviews, and understand plot scenes;

[0169] Based on the plot scenes of each video, combined with the frame-by-frame pages and subtitle files, the translation content of each language is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model, as described above.

[0170] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0171] In summary, the present invention provides a contextual translation processing method, device, intelligent device and storage medium based on a large model, including: obtaining voice information to be translated, obtaining scene information corresponding to the voice information; inputting the processed voice information and the corresponding scene information into a pre-trained scene-based translation model, and outputting the translation content corresponding to the scene; wherein the training steps of the pre-trained scene-based translation model include: collecting and obtaining different videos as training data; analyzing the frame-by-frame pages, obtaining subtitle files, describing and commenting on the video plot, and understanding the plot scenes of each video; based on the plot scenes of each video, combining the video frame-by-frame pages and the corresponding subtitle files, analyzing the corresponding translation content, training the scene translation model, and obtaining a pre-trained scene-based translation model. The translation of the present invention is not just a simple text translation, but also can be translated according to the scene in combination with the corresponding context, so that the translation is more accurate and the translation effect is better.

Claims

1. A contextual translation processing method based on a large model, characterized in that: include: When the translation function is detected to be turned on, the language of the associated translation is controlled, and the voice information to be translated is obtained, as well as the scene information corresponding to the voice information is obtained; Input the processed speech information and the corresponding scene information into a pre-trained scene-based translation model, and output the translation content corresponding to the scene; The training steps of the pre-trained scene-based translation model include: Collect different videos as training data; Analyze the frame-by-frame pages of the video to obtain corresponding subtitle files; and obtaining and analyzing the video plot descriptions and comments, and understanding the plot scenes of each video; Based on the plot scenes of each video, combined with the video frame-by-frame pages and corresponding subtitle files, the corresponding translation content is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model.

2. The contextual translation processing method based on a large model according to claim 1 is characterized in that: When the translation function is detected to be turned on, the step of controlling the language associated with the translation includes: Determining whether a translation request instruction is detected; When a translation request instruction is detected, the control starts the translation function and provides an interface for selecting a target language for translation; Receive user's operation instructions to confirm the target language to be translated, and control the associated translation language.

3. The contextual translation processing method based on a large model according to claim 2 is characterized in that: The steps of obtaining the voice information to be translated and obtaining the scene information corresponding to the voice information include: Based on the determined target language of translation, the voice information to be translated is extracted from the video file or voice file to be translated; The voice information that needs to be translated is identified and converted into translatable text content through voice signal processing and natural language processing technology; According to the geographical location scene, conversation content and context information of the video file or voice file to be translated, the specific scene of the video file or voice file to be translated is obtained.

4. The contextual translation processing method based on a large model according to claim 3 is characterized in that: The geographical location scene is the actual location where the video or voice content occurs; The dialogue content includes the subject, emotion, and role relationship involved in the dialogue; The context information includes the conversation content and the preceding and following context information.

5. The contextual translation processing method based on a large model according to claim 1 is characterized in that: The step of inputting the processed voice information and the corresponding scene information into a pre-trained scene-based translation model and outputting the translation content corresponding to the scene includes: The speech information to be translated is converted into text through speech recognition, and the scene information corresponding to the speech information is used as the input of the model and input into the pre-trained scene-based translation model; The pre-trained scenario-based translation model analyzes the original text according to the input text and scenario information, and generates a translation result in combination with the tone, wording and grammatical structure of the scenario.

6. The contextual translation processing method based on a large model according to claim 1 is characterized in that: The steps of obtaining the video plot description and comment content, analyzing the video plot description and comment content, and understanding the plot scenes of each video include: Collect a plot summary of the film's overall plot, as well as review content from audience or critics; Understand the context of the video and conduct in-depth analysis of the plot, characters, and possible emotional tone based on the collected plot summary description of the overall plot of the movie and the comments fed back by the audience or critics.

7. The contextual translation processing method based on a large model according to claim 1 is characterized in that: The training step of the pre-trained scene-based translation model further includes: Collect and download different movie videos as training data; Analyze the frame-by-frame pages of the video and obtain the corresponding movie subtitle files; Search movies online on Douban interface, obtain movie plot descriptions and user reviews, analyze plot descriptions and user reviews, and understand plot scenes; Based on the plot scenes of each video, combined with the frame-by-frame pages and subtitle files, the translation content of each language is analyzed, and the scene translation model is trained to obtain a pre-trained scene-based translation model.

8. A contextual translation processing device based on a large model, characterized in that: The device comprises: A pre-training module is used to collect and obtain different videos as training data; analyze the frame-by-frame pages of the videos to obtain corresponding subtitle files; and obtain the description and comment content of the video plots, analyze the description and comment content of the video plots, and understand the plot scenes of each video; based on the plot scenes of each video, combined with the frame-by-frame pages of the videos and the corresponding subtitle files, analyze the corresponding translation content, train the scene translation model, and obtain a pre-trained scene-based translation model; A translation acquisition module, which is used to control the language of the associated translation when detecting that the translation function is turned on, and to obtain the voice information to be translated, and to obtain the scene information corresponding to the voice information; The translation application module is used to input the processed voice information and the corresponding scene information into a pre-trained scene-based translation model and output the translation content corresponding to the scene.

9. A smart device, characterized in that: The device comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include being used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as described in any one of claims 1 to 7.

Images