Multi-language translation method and device based on selling scene, electronic equipment and storage medium

Abstract

The invention provides a multi-language real-time translation method based on a selling scene. The multi-language real-time translation method comprises the steps of obtaining first language information and second language information; the data processing module constructs a first system cue word and a second system cue word in real time according to the first language information and the second language information; obtaining first voice input information and second voice input information according to the first system prompt word and the second system prompt word; sending a preset first identifier, a preset second identifier, the first voice input information and the second voice input information to a multi-modal model to obtain a first translation result and a second translation result; and displaying the first translation result and the second translation result on a display module. According to the technical scheme of the embodiment, by obtaining the language data of the retail scene in real time, designing the multi-modal model of the training voice / text and optimizing the specific scene, translation results of different languages are displayed to the display module in a streaming mode in real time, and then the high-precision use requirement is met.

Description

Technical Field

[0001] The present invention relates to the field of real-time translation technology, and in particular to a multi-language translation method, device, equipment and medium based on a sales scenario. Background Art

[0002] The existing general streaming translation applications, that is, translation of general languages ​​for general scenarios, lack real-time translation applications for specific scenarios. In addition, the use of general algorithms in multi-language translation in specific sales scenarios cannot meet the high-precision requirements of specific scenarios. In addition, the multi-language translation process requires the training of corresponding two-way intercommunication translation models. Data of different modalities cannot supervise and enhance each other, and the process is relatively complicated. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a multi-language translation method, device, electronic device, and storage medium based on a sales scenario, which can meet the high-precision requirements of specific scenarios and perform real-time translation of different languages.

[0004] In a first aspect, an embodiment of the present invention provides a multi-language translation method based on a sales scenario, which is applied to a retail device, wherein the retail device includes a display module, a data processing module and a pickup module, wherein the data processing module is connected to the display module and the pickup module respectively, and the display module is connected to the pickup module. The method includes:

[0005] The pickup module acquires first language information and second language information;

[0006] The data processing module constructs a first system prompt word and a second system prompt word in real time according to the first language information and the second language information;

[0007] Acquire first voice input information and second voice input information according to the first system prompt word and the second system prompt word;

[0008] Sending a preset first identifier and a second identifier, as well as the first voice input information and the second voice input information to a multimodal model to obtain a first translation result and a second translation result;

[0009] The first translation result and the second translation result are displayed on the display module.

[0010] In some embodiments of the present invention, the pickup module includes a first pickup and a second pickup. Before the pickup module acquires the first language information and the second language information, the method further includes:

[0011] The first microphone acquires third voice input information;

[0012] The data processing module performs recognition according to the third voice input information to determine the language of the third voice input information;

[0013] When the data processing module cannot recognize the third voice input information, in response to a language selection instruction input by the user, the first language information of the user is acquired from the language selection instruction.

[0014] In some embodiments of the present invention, the data processing module constructs a first system prompt word and a second system prompt word in real time according to the first language information and the second language information, including:

[0015] determining a task conversion type of the first language information and the second language information;

[0016] The target language information is acquired, the first language information and the second language information are converted according to the task conversion type, and the first voice input information and the second voice input information are converted into a text sequence corresponding to the target language information.

[0017] In some embodiments of the present invention, the sending to the multimodal model includes:

[0018] Inputting the first voice input information, the second voice input information, the first system prompt word and the second system prompt word into a preset text dictionary through a vocabulary scanner;

[0019] A multilingual data set is obtained, and the text dictionary is trained according to a preset first algorithm and the multilingual data set to update the first conversion result and the second conversion result in real time.

[0020] In some embodiments of the present invention, after obtaining the first voice input information and the second voice input information according to the first system prompt word and the second system prompt word, the method further includes:

[0021] Obtain scene language data;

[0022] combining the scene language data, the first language information, the second language information, the first system prompt word, and the second system prompt word into a plurality of sequence data;

[0023] The plurality of sequence data are used as training inputs of the multimodal model.

[0024] In some embodiments of the present invention, the display module includes a first display screen and a second display screen, the first display screen and the second display screen are both connected to the data processing module, and the displaying the first conversion result and the second conversion result on the display module includes:

[0025] Displaying the first translation result on the first display screen, and displaying the second translation result on the second display screen;

[0026] When the pickup module cannot recognize the second language, the multimodal model outputs an unrecognizable result to the first display screen and the second display screen, or the data processing module controls the first display screen and the second display screen to display recognition failed entries.

[0027] In some embodiments of the present invention, sending the preset first identifier and the second identifier, and the first voice input information and the second voice input information to the multimodal model includes:

[0028] combining the first voice information, the first identifier, and the second language information into a first translation request, so that the multimodal model outputs the first translation result according to the first translation request;

[0029] The second voice information, the second identifier and the first language information are combined into a second translation request, so that the multimodal model outputs the second translation result according to the second translation request.

[0030] In a second aspect, an embodiment of the present invention provides a multi-language real-time translation device based on a sales scenario, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the multi-language real-time translation method based on the sales scenario as described in the first aspect above.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device, comprising the multi-language real-time translation device based on the sales scenario as described in the second aspect above.

[0032] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the multi-language real-time translation method based on the sales scenario as described in the first aspect above.

[0033] The multi-language real-time translation method based on the sales scenario according to the embodiment of the present invention has at least the following beneficial effects:

[0034] The microphone module obtains the first language information and the second language information; the data processing module constructs the first system prompt word and the second system prompt word in real time according to the first language information and the second language information; obtains the first voice input information and the second voice input information according to the first system prompt word and the second system prompt word; sends the preset first identification and the second identification, as well as the first voice input information and the second voice input information to the multimodal model to obtain the first translation result and the second translation result; displays the first translation result and the second translation result on the display module. According to the technical solution of this embodiment, voice is captured by the microphone module, the language data of the retail scene is obtained in real time, and a multimodal model of voice / text is designed and trained. Through the mutual supervision of voice and text, it is optimized for specific scenes, so that the translation results of different languages ​​can be streamed to the display module in real time, thereby meeting the high-precision usage requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flowchart of a multi-language translation method based on a sales scenario provided by an embodiment of the present invention;

[0036] Figure 2 This is a flow chart before the pickup module provided by the present invention obtains the first language information and the second language information;

[0037] Figure 3 It is a flow chart of the data processing module provided by the present invention constructing the first system prompt word and the second system prompt word in real time according to the first language information and the second language information;

[0038] Figure 4 It is a flow chart of sending to a multimodal model for training provided by the present invention;

[0039] Figure 5 is a flow chart after obtaining the first voice input information and the second voice input information according to the first system prompt word and the second system prompt word provided by the present invention;

[0040] Figure 6 is a flow chart of displaying the first conversion result and the second conversion result on the display module provided by the present invention;

[0041] Figure 7 It is a flowchart provided by the present invention for sending a preset first identifier and a second identifier, as well as first voice input information and second voice input information to a multimodal model for training;

[0042] Figure 8 is a schematic diagram of the structure of the retail device provided by the present invention;

[0043] Fig. 9 It is a structural diagram of a multi-language translation device based on a sales scenario provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0044] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0045] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0046] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0047] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0048] Reference Fig. 9The embodiment of the present invention provides a multi-language translation method based on a sales scenario, which is applied to a retail device. The retail device includes a display module 100, a data processing module 200 and a pickup module 300. The data processing module 200 is connected to the display module 100 and the pickup module 300 respectively. The display module 100 includes a first display screen 110 and a second display screen 220. The pickup module 300 includes a first pickup 310 and a second pickup 320. The first display screen 110 is connected to the first pickup 310, and the second display screen 120 is connected to the second pickup 320. The first pickup 310 is used to obtain the user's first language information in real time, and the second pickup 310 is used to obtain the user's first language information in real time. The speaker 320 is used to obtain the second language information of the salesperson in real time. A multimodal model is preset in the data processing module to recognize and translate the first language information and the second language information in real time. The multimodal model is optimized for specific sales scenarios through mutual supervision of voice and text to achieve high-precision usage requirements. Furthermore, this embodiment is also provided with an end-to-end real-time voice translation process based on a retail scenario, by constructing a first system prompt word and a second system prompt word, and using the preset first identifier and the second identifier as the input of the multimodal model, so that the multimodal model can perform recognition conversion and stream the first translation result and the second translation result on the first display screen and the second display screen.

[0049] The control method of the embodiment of the present invention is further described below based on the accompanying drawings.

[0050] Reference Figure 1 , Figure 1 A flowchart of a multi-language translation method based on a sales scenario provided by an embodiment of the present invention, the multi-language translation method based on a sales scenario includes but is not limited to the following steps:

[0051] Step S11, the pickup module obtains the first language information and the second language information;

[0052] It should be noted that the first language information is the language of the user, and the second language information is the language used by the salesperson, wherein the first language information and the second language information can be the same or different, and this embodiment does not make specific restrictions. The pickup module captures and distinguishes language information of different languages ​​in real time, and inputs the obtained first language information and second language information into the data processing module for further translation processing.

[0053] Furthermore, the second language information can be set by oneself, and the second language information can be a language commonly used by the salesperson.

[0054] Step S12, the data processing module constructs the first system prompt word and the second system prompt word in real time according to the first language information and the second language information;

[0055] It should be noted that the data processing module retrieves the corresponding text content from a preset text dictionary according to the current interactive context and user needs, and constructs a first system prompt word according to the recognized first language information, and constructs a second system prompt word according to the second language information, so that the retail device can support multiple languages ​​and be expanded according to the practical needs of users.

[0056] Step S13, obtaining first voice input information and second voice input information according to the first system prompt word and the second system prompt word;

[0057] It should be noted that the first system prompt word is combined with the first language information so that the data processing module can obtain the complete first voice input information, and the second system prompt word is combined with the second language information so that the data processing module can obtain the complete second voice output information. By combining the system prompt word language information, the data processing module can accurately recognize and understand the user's voice output content, thereby improving the accuracy and reliability of the real-time translation process.

[0058] Step S14, sending the preset first identifier and the second identifier, as well as the first voice input information and the second voice input information to the multimodal model to obtain a first translation result and a second translation result;

[0059] It should be noted that the multimodal model can process and fuse data of different modalities, such as text and voice. In the retail scenario, the multimodal model accepts the first voice input information and the second voice input information as data of the voice modality, and uses the first identifier and the second identifier as auxiliary information, wherein the first identifier and the second identifier include but are not limited to language type, sales scenario and user identity. The multimodal model further improves the real-time translation accuracy of the data processing module by fusing the voice input information and the language text.

[0060] In addition, the multimodal model will perform feature extraction on the first voice input information and the second voice input information, converting the original audio signal into numerical features that can be understood by the data processing module. At the same time, the multimodal model will also perform fusion processing on the first identifier and the second identifier to extract the corresponding translation features.

[0061] Step S15: display the first translation result and the second translation result on a display module.

[0062] It should be noted that the first translation result and the second translation result converted by the multimodal model are stream-displayed in the display module. Specifically, the second display screen on the salesperson side displays the first translation result on the user side, and the first display screen on the user side displays the second translation result on the salesperson side. By simultaneously displaying the first translation result and the second translation result, the user and the salesperson can obtain the language translation of each other in real time, especially in the sales scenario when the user and the salesperson do not speak the same language, which improves the flexibility of translation.

[0063] It should be noted that the retail scene has the following language characteristics: the customer group is uncertain, and the same retail machine may contact customers of different countries and languages, that is, the same machine needs to support voice conversion in multiple languages; the retail scene is single, the terminology is basically the same, and the general language model has a poor effect in the corresponding scene. It is necessary to set the value and fine-tune the multi-language model in the machine, which makes it impossible for both parties to perform effective real-time translation. This embodiment recognizes and translates the first language information and the second language information in real time. The multimodal model optimizes the specific sales scene through mutual supervision of voice and text to achieve high-precision usage requirements. Furthermore, this embodiment is also provided with an end-to-end real-time voice translation process based on the retail scene, by constructing the first system prompt word and the second system prompt word, and using the preset first identifier and the second identifier as the input of the multimodal model, so that the multimodal model can perform recognition and conversion and stream the first translation result and the second translation result on the first display screen and the second display screen, thereby completing the real-time translation of both parties in the sales scene.

[0064] In addition, in one embodiment, referring to Figure 2 ,exist Figure 1 Before step S11 of the illustrated embodiment, the following steps are also included but not limited to:

[0065] Step S21, the first microphone obtains third voice input information;

[0066] Step S22, the data processing module performs recognition based on the third voice input information to determine the language of the third voice input information;

[0067] Step S23: when the data processing module cannot recognize the third voice input information, in response to the language selection instruction input by the user, the first language information of the user is obtained from the language selection instruction.

[0068] It should be noted that the third voice input information is the voice input information of the user captured by the first microphone for the first time. After the first microphone receives the voice information sent by the user, the data processing module determines the language of the user. When the data processing module cannot recognize the language of the user, the user is prompted to enter a language selection instruction. The language selection instruction is usually operated by the user through the first display (such as clicking a button or a drop-down menu selection, or voice output under the premise of supporting the language). After the language selection instruction output by the user is replenished by the data processing module, the language information specified by the user (i.e., the first language information) is extracted. In this embodiment, by allowing the user to manually select the language, even if the data processing module cannot effectively determine the language, the user can manually select the language based on the use of the vending device, thereby avoiding the situation where the inability to effectively purchase due to language mismatch occurs, further improving the user experience.

[0069] In addition, in one embodiment, referring to Figure 3 ,exist Figure 1 Step S12 of the illustrated embodiment also includes but is not limited to the following steps:

[0070] Step S31, determining the task conversion type of the first language information and the second language information;

[0071] Step S32, obtaining target language information, converting the first language information and the second language information according to the task conversion type, and converting the first voice input information and the second voice input information into a text sequence corresponding to the target language information.

[0072] It should be noted that, based on the first language information (user) and the second language information (salesperson) captured by the pickup module, after determining the first language information and the second language information, the data processing module converts the first language information and the second language information into text information, i.e., the first language text and the second language text, and performs semantic analysis on the first language text and the second language text. For example, this task type is speech-to-text, the input is speech, and the output is a text sequence corresponding to the target language. By converting the language used by the user and the language used by the salesperson into each other according to the recognized text sequence, the flexibility and accuracy of information in different languages ​​in retail scenarios can be effectively improved.

[0073] In addition, in one embodiment, referring to Figure 4 ,exist Figure 1 Step S13 of the illustrated embodiment also includes but is not limited to the following steps:

[0074] Step S41, inputting the first voice input information, the second voice input information, the first system prompt word and the second system prompt word into a preset text dictionary through a vocabulary scanner;

[0075] Step S42, obtaining a multilingual data set, and training the text dictionary according to a preset first algorithm and the multilingual data set to update the first conversion result and the second conversion result in real time.

[0076] It should be noted that the first voice input information, the second voice input information, the first system prompt word and the second system prompt word are all input into a unified text dictionary through a vocabulary scanner, and the text dictionary is trained and expanded on a preset multilingual data set based on the SentencePiece algorithm (i.e., the first algorithm), thereby obtaining a text word list. Furthermore, the first voice input information and the second voice input information are sampled and discretized based on a certain frequency, and the unified voice and text word lists are used for data preparation and input into the training process of the multimodal model.

[0077] It should be noted that this embodiment adopts the LLaMA architecture as a whole, performs Autogressive training through the first voice input information, the second voice input information and the text sequence, predicts the next text sequence, and thus performs supervision, and performs gradient update through the CrossEntropy cross entropy loss.

[0078] In addition, in one embodiment, referring to Figure 5 ,exist Figure 1 After step S13 in the illustrated embodiment, the following steps are also included but not limited to:

[0079] Step S51, obtaining scene language data;

[0080] Step S52, combining the scene language data, the first language information, the second language information, the first system prompt word and the second system prompt word into a plurality of sequence data;

[0081] Step S53: using the multiple sequence data as training input for the multimodal model.

[0082] It should be noted that according to the scale of tourism industry in different usage areas, scene language data of commonly used languages ​​such as Chinese, English, Japanese, German and Thai are collected to train the multimodal model in language or text, and any reasonable sequence data is used as the training input of the multimodal model. Specifically, the format of the sequence data is the first system prompt word of the first language information under the scene language data + the second system prompt word of the second language information under the scene language data; the first system prompt word of the second language information under the scene language data + the second system prompt word of the first language information under the scene language data set..., wherein the first system prompt word and the second system prompt word can be the same language or different languages, and the sequence data are randomly combined during training.

[0083] In addition, in one embodiment, referring to Figure 6 ,exist Figure 1 Step S15 of the illustrated embodiment also includes but is not limited to the following steps:

[0084] Step S61, displaying the first translation result on the first display screen, and displaying the second translation result on the second display screen;

[0085] Step S62, when the pickup module cannot recognize the second language, the multimodal model outputs the unrecognized result to the first display screen and the second display screen, or the data processing module controls the first display screen and the second display screen to display the recognition failed words.

[0086] It should be noted that when the user inputs information in a first language (such as English) through voice or text in a sales scenario, the first language information is translated into second language information (such as Chinese) according to a preset text dictionary through the multimodal model, and the second language information is sent to the second display screen as the first translation result, so that the salesperson can directly see the translated content; or the second language information (such as Chinese) is input through voice or text, and the second language information is translated into first language information (such as English) through the multimodal model according to a preset text dictionary, and the first language information is sent to the first display screen as the second translation result, so that the user can see the translated content.

[0087] Furthermore, when the pickup module cannot recognize the second language, the data processing module directly controls the first display screen and the second display screen to display words such as "failed" or "unrecognizable" to quickly let the user understand the current status and take corresponding countermeasures, such as re-entering or changing the language.

[0088] In addition, in one embodiment, referring to Figure 7 ,exist Figure 1 Step S14 of the illustrated embodiment also includes but is not limited to the following steps:

[0089] Step S71, combining the first voice information, the first identifier and the second language information into a first translation request, so that the multimodal model outputs a first translation result according to the first translation request;

[0090] Step S72: Combine the second voice information, the second identifier and the first language information into a second translation request, so that the multimodal model outputs a second translation result according to the second translation request.

[0091] It should be noted that when the data processing module receives the first voice message from the user, it also records the first identifier associated with the first voice message (including but not limited to the user ID, session ID or other unique identifiers used to distinguish different requests), and the data processing module integrates the first voice message, the first identifier and the target language specified by the user into a complete translation request, i.e., the first translation request. The same is true for the second translation result.

[0092] After the first translation request and the second translation request are processed by the multimodal model, the obtained first translation result and the second translation result are sent to the first display and the second display for display respectively. During the translation process, the user only needs to perform simple voice input to achieve multi-language translation, and the data processing module can return the language translation results of the user and the salesperson in real time, so that the user and the salesperson can communicate in real time and smoothly. This breaks the language barrier between different languages.

[0093] like Figure 8 As shown, Figure 8 : is a structural diagram of a multi-language real-time translation device based on a sales scenario provided by an embodiment of the present invention. The present invention also provides a multi-language real-time translation device based on a sales scenario, including:

[0094] The processor 901 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0095] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the multi-language real-time translation method based on the sales scenario in the embodiment of this application;

[0096] Input / output interface 903, used to implement information input and output;

[0097] Communication interface 904, used to realize communication interaction between the present apparatus and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WI FI, Bluetooth, etc.);

[0098] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0099] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0100] An embodiment of the present application also provides an electronic device, including the multi-language real-time translation device based on the sales scenario as described above.

[0101] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the multi-language real-time translation method based on the sales scenario is implemented.

[0102] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0104] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A multilingual translation method based on sales scenarios, characterized in that: Applied in a retail device, the retail device comprises a display module, a data processing module and a pickup module, the data processing module is connected to the display module and the pickup module respectively, the display module is connected to the pickup module, the method comprises: The pickup module acquires first language information and second language information; The data processing module constructs a first system prompt word and a second system prompt word in real time according to the first language information and the second language information; Acquire first voice input information and second voice input information according to the first system prompt word and the second system prompt word; Sending a preset first identifier and a second identifier, as well as the first voice input information and the second voice input information to a multimodal model to obtain a first translation result and a second translation result; The first translation result and the second translation result are displayed on the display module.

2. The multilingual translation method based on sales scenarios according to claim 1, characterized in that: The pickup module includes a first pickup and a second pickup. Before the pickup module acquires the first language information and the second language information, the method further includes: The first microphone acquires third voice input information; The data processing module performs recognition according to the third voice input information to determine the language of the third voice input information; When the data processing module cannot recognize the third voice input information, in response to a language selection instruction input by the user, the first language information of the user is acquired from the language selection instruction.

3. The multilingual translation method based on sales scenarios according to claim 1, characterized in that: The data processing module constructs a first system prompt word and a second system prompt word in real time according to the first language information and the second language information, including: determining a task conversion type of the first language information and the second language information; The target language information is acquired, the first language information and the second language information are converted according to the task conversion type, and the first voice input information and the second voice input information are converted into a text sequence corresponding to the target language information.

4. The multilingual translation method based on sales scenarios according to claim 1, characterized in that: The sending to the multimodal model includes: Inputting the first voice input information, the second voice input information, the first system prompt word and the second system prompt word into a preset text dictionary through a vocabulary scanner; A multilingual data set is obtained, and the text dictionary is trained according to a preset first algorithm and the multilingual data set to update the first conversion result and the second conversion result in real time.

5. The multi-language real-time translation method based on sales scenarios according to claim 1, characterized in that: After acquiring the first voice input information and the second voice input information according to the first system prompt word and the second system prompt word, the method further includes: Obtain scene language data; combining the scene language data, the first language information, the second language information, the first system prompt word, and the second system prompt word into a plurality of sequence data; The plurality of sequence data are used as training inputs of the multimodal model.

6. The multi-language real-time translation method based on sales scenarios according to claim 1, characterized in that: The display module includes a first display screen and a second display screen, and the first display screen and the second display screen are both connected to the data processing module. The first conversion result and the second conversion result are displayed on the display module, including: Displaying the first translation result on the first display screen, and displaying the second translation result on the second display screen; When the pickup module cannot recognize the second language, the multimodal model outputs an unrecognizable result to the first display screen and the second display screen, or the data processing module controls the first display screen and the second display screen to display recognition failed entries.

7. The multi-language real-time translation method based on sales scenarios according to claim 1, characterized in that: The sending the preset first identifier and the second identifier, as well as the first voice input information and the second voice input information to the multimodal model includes: combining the first voice information, the first identifier, and the second language information into a first translation request, so that the multimodal model outputs the first translation result according to the first translation request; The second voice information, the second identifier and the first language information are combined into a second translation request, so that the multimodal model outputs the second translation result according to the second translation request.

8. A multi-language real-time translation device based on a sales scenario, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the multi-language real-time translation method based on the sales scenario as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: Including the multi-language real-time translation device based on sales scenarios as described in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the multi-language real-time translation method based on sales scenarios as described in any one of claims 1 to 7.

Images