Synchronous processing method and system for instant messaging message

Abstract

The invention discloses a synchronous processing method and system for instant messaging messages, and relates to the technical field of telecommunication, and the method comprises the following steps: obtaining preset language systems of a first user and a second user, and judging whether the language systems are consistent or not when the first user and the second user establish communication; if not, the message sent by the second user is converted into the language system of the first user and then sent to the terminal of the first user, and the message sent by the first user is converted into the language system of the second user and then sent to the terminal of the second user, so that the language synchronization of instant messaging is realized. The technical problem that an existing instant messaging tool is insufficient in efficiency and accuracy in multi-language real-time conversion is solved, and the technical effect of efficient and accurate cross-language instant messaging is achieved.

Description

Technical Field

[0001] The present application relates to the field of electrical communication technology, and in particular to a method and system for synchronously processing instant messaging messages. Background Art

[0002] In the context of globalization, the demand for communication between users of different languages ​​is increasing, especially in business and social instant messaging, where cross-language communication has become a universal demand. In existing instant messaging technologies, although some tools provide translation functions, they usually rely on external translation applications or plug-ins, and cannot achieve efficient and seamless real-time translation. In addition, these systems are often limited to simple text conversion when processing voice information, and lack the processing of voice features and emotions, which affects the semantic accuracy and emotional consistency of information transmission. Current technology also lacks a multi-user, cross-language synchronous processing mechanism, and the user experience is not smooth enough. Therefore, there is an urgent need for a technical solution that can automatically identify the user's language system, realize real-time conversion between text and voice, and maintain consistency in information semantics and emotional expression, so as to improve the naturalness and immediacy of multilingual communication.

[0003] At present, there is a technical problem in the related technologies that instant messaging tools are not efficient and accurate enough in real-time conversion of multiple languages. Summary of the invention

[0004] The present application provides a method and system for synchronously processing instant messaging messages, which obtains the preset language system of the first user and the second user, and determines whether their language systems are consistent when the two establish communication. If they are different, the message sent by the second user is converted into the language system of the first user and sent to the terminal of the first user, and the message sent by the first user is converted into the language system of the second user and sent to the terminal of the second user, thereby realizing language synchronization of instant messaging and achieving the technical effect of realizing efficient and accurate cross-language instant messaging.

[0005] The present application provides a method for synchronously processing instant messaging messages, comprising:

[0006] Obtaining a preset language system of a first user and a preset language system of a second user; when the first user establishes instant messaging with the second user, determining whether the preset language system of the first user is consistent with the preset language system of the second user; if the preset language system of the first user is different from the preset language system of the second user, performing language conversion on a first instant messaging message sent by the second user according to the preset language system of the first user, and obtaining first instant conversion language information, wherein the first instant conversion language information is language information after the first instant messaging message is converted from the preset language system of the second user to the preset language system of the first user; sending the first instant conversion language information to a communication terminal of the first user, and simultaneously performing language conversion on a second instant messaging message sent by the first user, and obtaining second instant conversion language information, wherein the second instant conversion language information is language information after the second instant messaging message is converted from the preset language system of the first user to the preset language system of the second user; sending the second instant conversion language information to the communication terminal of the second user, so as to establish instant messaging between the first user and the second user.

[0007] The present application also provides a system for synchronously processing instant messaging messages, including:

[0008] a preset language system acquisition module, the preset language system acquisition module is used to obtain the preset language system of the first user and the preset language system of the second user; a preset language system judgment module, the preset language system judgment module is used to determine whether the preset language system of the first user is consistent with the preset language system of the second user when the first user establishes instant messaging with the second user; a first instant conversion language information acquisition module, the first instant conversion language information acquisition module is used to convert the language of the first instant messaging message sent by the second user according to the preset language system of the first user if the preset language system of the first user is different from the preset language system of the second user, and obtain the first instant conversion language information, the first instant conversion language information is The first instant messaging message is converted from the preset language system of the second user to the language information of the preset language system of the first user; a language conversion module, the language conversion module is used to send the first instant conversion language information to the communication terminal of the first user, and at the same time perform language conversion on the second instant messaging message sent by the first user to obtain second instant conversion language information, the second instant conversion language information is the language information of the second instant messaging message converted from the preset language system of the first user to the preset language system of the second user; an instant messaging establishment module, the instant messaging establishment module is used to send the second instant conversion language information to the communication terminal of the second user to establish instant messaging between the first user and the second user.

[0009] The instant messaging message synchronization processing method and system proposed in this application first obtains the preset language system of the first user and the second user, and determines whether their language systems are consistent when the two establish communication. If they are different, the message sent by the second user is converted into the language system of the first user and sent to the terminal of the first user, and the message sent by the first user is converted into the language system of the second user and sent to the terminal of the second user, thereby realizing the language synchronization of instant messaging and achieving the technical effect of realizing efficient and accurate cross-language instant messaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0011] Figure 1 A schematic diagram of a flow chart of a method for synchronously processing instant messaging messages provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of the structure of an instant messaging message synchronization processing system provided in an embodiment of the present application.

[0013] Description of reference numerals: preset language system acquisition module 10 , preset language system determination module 20 , first instant conversion language information acquisition module 30 , language conversion module 40 , instant messaging establishment module 50 . DETAILED DESCRIPTION

[0014] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0015] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0016] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0017] The present application embodiment provides a method for synchronously processing instant messaging messages, such as Figure 1 As shown, the method includes:

[0018] Step S100, obtain the preset language system of the first user and the preset language system of the second user. Specifically, when the user registers with the instant messaging system for the first time, a friendly and intuitive registration interface will be presented. In addition to basic information, a language setting guide section is provided. Common languages ​​such as English and Chinese can be selected through a drop-down menu. It also supports inputting international standard language codes such as ISO639-1 to specify languages. For those who have dialect preferences, further selections can be made, such as Mandarin and Cantonese in Chinese. The system can also intelligently recommend languages ​​based on the initial behaviors such as the regional network access point and keyboard input method when the user registers. After the user completes the selection, the language setting information is encrypted and stored in a database with powerful backup and recovery functions, which is closely associated with the user account. Each account has a special data table item to store language setting data. The user can modify the language in the setting menu during use. After the modification, the system updates the database information in real time. Through these steps, the system can accurately obtain the preset language system of the first user and the second user.

[0019] In a possible implementation, the preset language system of the first user and the preset language system of the second user are obtained, and step S100 further includes step S110, when there are more than two users participating in establishing the instant messaging, identifying the preset language system of each user. Specifically, when the number of users participating in establishing the instant messaging exceeds two, the system needs to identify the preset language system of each user. During the user registration stage, each user has set his or her own preset language system. The system stores the information in a secure database and associates it with the user account. The system will traverse all user accounts participating in the group chat and extract the preset language system information of each user from the database, which is the basis for the subsequent establishment of an accurate information conversion channel to ensure that the system clearly understands the language form in which each user expects to receive information.

[0020] Step S120, taking a single participating user as a target, establishing multiple conversion channels according to the preset language system of each user, wherein the multiple conversion channels include information conversion channels of each preset language system between the target user and other communicating users. Specifically, for each participating user, take it as a target user, and establish multiple conversion channels according to the preset language system of all users. For example, in a group chat with users A, B, and C, if user A is the target user, it is necessary to establish information conversion channels between user A and user B, and between user A and user C. Since each user has his own preset language system, the channels must be able to achieve conversion between different language systems. Specifically, if the preset language system of user A is English, the preset language system of user B is French, and the preset language system of user C is Chinese, then the conversion channel from user B to user A must be able to convert French information into English information, and the conversion channel from user C to user A must be able to convert Chinese information into English information. For each target user, such a complete set of conversion channels of different language systems between other communicating users must be established to ensure that information can be accurately circulated between users of different language systems.

[0021] Step S130, obtain the instant messaging information sent by the non-target user in the instant messaging, perform language system conversion through the multiple conversion channels respectively, generate the instant messaging information of the target language system, and send it to the communication terminal of the target user. Specifically, during the group chat process, the system continuously obtains the instant messaging information. When a non-target user sends an instant messaging information, for example, in the above example, user B or user C sends a message, the system will convert the message through the corresponding multiple conversion channels for the target user A. If user B sends a French message, this message will be converted into an English message through the conversion channel from user B to user A according to the preset language conversion rules and algorithms; similarly, if user C sends a Chinese message, it will also be converted into an English message through the corresponding channel. After conversion, the instant messaging information that conforms to the language system of the target user A is generated, and then the converted information is accurately sent to the communication terminal of the target user A. For each user in the group chat as a target user, the same operation is performed, thereby realizing smooth instant messaging between users of different language systems in the group chat.

[0022] In a possible implementation, the instant messaging information sent by non-target users in the instant messaging is obtained, and the language system is converted through the multi-conversion channel respectively to generate the instant messaging information of the target language system, and sent to the communication terminal of the target user. Step S130 further includes step S131, and the instant messaging information includes: text information and voice information. Specifically, in instant messaging, the text information is input by the user using the device input interface (such as a mobile phone virtual keyboard, a computer keyboard, etc.), and the operating system collects the character data and integrates it into text. After being converted into a byte stream by encoding standards such as UTF-8 and UTF-16 and adding a protocol header and metadata (identifying the information type, source, recipient, etc.), it is transmitted using network protocols such as TCP / IP, and after being received by the receiving end network module, it is decoded by the instant messaging application and displayed in the chat window. Voice information is the sound vibration converted by the microphone into an analog electrical signal, which is then converted into a digital signal through sampling (taking values ​​at certain time intervals) and quantization (converting the sampled values ​​into digital codes). It is then encoded and compressed using encoding algorithms such as MP3, AAC, Opus, and after adding protocol headers and metadata, it is transmitted through a network protocol (which may use streaming media transmission). After receiving it, the receiving end decodes and decompresses it into a digital voice signal, which is finally converted into an analog audio signal by the audio playback system and played through the speaker, thereby meeting the diverse communication needs of users.

[0023] Step S200, when the first user establishes instant messaging with the second user, determine whether the preset language system of the first user is consistent with the preset language system of the second user. Specifically, when the first user and the second user initiate an instant messaging connection request and successfully establish a connection, the system automatically triggers the language system consistency judgment program. First, obtain the preset language system information of the two from the database storing user data based on the user's unique identifier (such as user ID). If it is stored in a language code (such as ISO639-1), it is directly compared. If it is more complex (such as containing dialect information), it is further parsed. After the comparison is completed, if it is consistent, the message is directly transmitted and displayed; if it is inconsistent, the language conversion module is started to process the message according to the rules, providing an accurate and efficient processing basis for multi-language interaction.

[0024] Step S300: If the preset language system of the first user is different from the preset language system of the second user, the first instant messaging message sent by the second user is converted into a language according to the preset language system of the first user to obtain first instant conversion language information, wherein the first instant conversion language information is the language information after the first instant messaging message is converted from the preset language system of the second user into the preset language system of the first user. Specifically, when the preset language systems of the first user and the second user are different and the second user sends a first instant messaging message to the first user, the system receives the message and confirms the language systems of both. If the message is a text message, it is first subjected to grammatical analysis, and words, phrases and sentence components are identified according to the grammatical rules of its language (such as French), and semantics are mined with the help of a vocabulary library and a semantic relationship network. After that, vocabulary conversion is performed according to the preset language system of the first user (such as English) through a vocabulary mapping table or a conversion model, and then the sentence is reconstructed according to the grammatical rules of the language. If the message is in the form of speech, the speech is first recognized as text and then converted according to the text information processing process. If it needs to be converted into speech, the speech synthesis technology is used to generate the first instant conversion language information in the form of speech according to the speech characteristics of the language.

[0025] In a possible implementation, if the preset language system of the first user is different from the preset language system of the second user, the language of the first instant messaging message sent by the second user is converted according to the preset language system of the first user to obtain first instant conversion language information, the first instant conversion language information being the language information after the first instant messaging message is converted from the preset language system of the second user to the preset language system of the first user. Step S300 further includes step S310, when the instant messaging message is a text message, semantic information is obtained according to the first instant messaging message and its corresponding preset language system of the second user. Specifically, when the instant messaging message received by the system is a text message, it must first be deeply analyzed to obtain semantic information, which is based on the preset language system of the second user. Different languages ​​have their own unique language structures and grammatical rules. Taking the preset language system of the second user as French as an example, if the first instant messaging message is "Je vais à l'école", the system will process it based on French grammar and vocabulary knowledge, and recognize that "Je" is the subject "I", "vais" is the first-person conjugation of the verb "go", "à" means direction, and "l'école" is "school". Through the analysis of vocabulary and grammatical structure, combined with a large amount of language knowledge stored in the French language model, including the semantics of vocabulary, phrase collocation rules, sentence structure patterns, etc., the system can accurately parse the semantics of the entire sentence, that is, "I go to school", which involves complex natural language processing technologies such as lexical analysis and syntactic analysis. Lexical analysis will perform operations such as part-of-speech tagging and stem extraction on the words in the sentence, while syntactic analysis will determine the grammatical relationship between words, thereby constructing a grammatical tree for the sentence to assist in the understanding of semantics and consider contextual information. If there are related topics in the previous conversation, it will supplement and correct the understanding of the semantics of the current sentence.

[0026] Step S320: convert the semantic information into text using the preset language system of the first user to obtain instant converted language information after the language system conversion. Specifically, after obtaining the semantic information based on the preset language system of the second user, it is necessary to perform text conversion according to the preset language system of the first user. Assuming that the preset language system of the first user is English, for the semantics "I go to school" just obtained, the system will look for a suitable expression in the English language system. A large number of language mapping relationships are stored in the system, and the relationship is obtained based on bilingual dictionaries, corpus statistics or machine learning model training. In this case, "I" corresponds to "I" and "going to school" corresponds to "go to school". After such conversion, the sentence becomes "I go to school". The instant conversion language information after the language system conversion needs to ensure the accurate transmission of semantics during the conversion process, and also conform to the expression habits of the preset language system of the first user. For example, different languages ​​have different expressions for grammatical categories such as tense, voice, and mood. The system will perform correct conversion according to the specific semantics and grammatical rules of the target language. If there are words or grammatical structures representing future tense in the original French sentence, when converting it to English, the form of expressing future tense in English, such as "will go to school", should also be accurately used to ensure that the instant conversion language information after conversion can accurately convey the content of the original message in terms of grammar and semantics.

[0027] In a possible implementation, if the preset language system of the first user is different from the preset language system of the second user, the language of the first instant messaging message sent by the second user is converted according to the preset language system of the first user to obtain first instant conversion language information, where the first instant conversion language information is the language information after the first instant messaging message is converted from the preset language system of the second user to the preset language system of the first user. Step S300 further includes step S330, when the instant messaging message is language information, text extraction is performed according to the first instant messaging message and the preset language system of the second user to obtain instant messaging text information. Specifically, when the received instant messaging message is language information, the system first performs text extraction based on the preset language system of the first instant messaging message and the second user. The system uses a speech recognition model trained for the preset language system of the second user to process the voice information. For example, if the preset language system of the second user is English, the speech recognition model in the system is trained with a large amount of English voice data. When a voice message is received, the model analyzes the acoustic features in the voice signal, including pitch, intensity, timbre, resonance peaks, etc., and identifies the phoneme sequence in the voice by matching it with the English voice pattern stored in the model. According to the language rules and vocabulary of English, the phoneme sequence is combined into words, and the words are further combined into sentences, thereby completing the text extraction and obtaining the text information of the instant messaging. It is necessary to process complex speech phenomena such as connected reading, weak reading, and accent to ensure that the extracted text information is accurate. For example, for words such as "gonna" (a colloquial expression of going to) in English, the system can accurately identify and convert them into corresponding standard written forms, providing accurate input for subsequent translation operations.

[0028] Step S340, using the preset language system of the first user to translate the text information of the instant messaging, and obtain the converted text information of the instant messaging. Specifically, after obtaining the text information of the instant messaging, the system translates it according to the preset language system of the first user. The system stores a large number of bilingual or multilingual corpora and language models. For the extracted text information, grammatical analysis is first performed to determine the sentence structure and part of speech information. For example, if the extracted text information is "I'm going to the park", the system will recognize that "I" is the subject, "'m going to" is the predicate, indicating "will go" and "the park" is the object. According to the preset language system of the first user (assuming it is Chinese), each word and phrase is translated by searching for the corresponding translation in the corpus or using a translation model based on a neural network. "I" is translated as "I", "'m going to" is translated as "plan to go", and "the park" is translated as "park", thus obtaining the converted instant messaging text message "I plan to go to the park". In the translation process, factors such as grammatical differences, lexical ambiguity and cultural background between different languages ​​need to be taken into consideration to ensure the accuracy and fluency of the translation.

[0029] Step S350, sending the converted instant messaging text message to the user's communication terminal, or performing voice generation on the converted instant messaging text message, and sending the converted language information to the user's communication terminal. Specifically, after completing the translation, the system can directly send the converted instant messaging text message to the user's communication terminal. The communication terminal can be an instant messaging application interface on a mobile phone, computer or other device. The system will encapsulate the text message according to the instant messaging protocol, transmit it to the target user's device through the network, and display it in the corresponding chat window, so that the user can directly read the translated content. If the user sets to receive voice messages or the system decides to generate voice according to the default settings, the system will perform voice generation on the converted instant messaging text message. The system will analyze the language characteristics of the converted text message. For example, for Chinese, we need to consider factors such as the pronunciation, tone, and rhythm of Chinese characters. According to the characteristics of the language, we select appropriate speech units from the speech synthesis library. Speech units can be syllables, phonemes, or other acoustic fragments. By splicing speech units and adjusting their acoustic parameters (such as pitch, volume, speaking speed, etc.), a natural and smooth speech waveform is generated. For example, for the sentence "I plan to go to the park", the system will synthesize the corresponding Chinese speech to make it sound natural and clear. Finally, the generated voice information (converted language information) is sent to the user's communication terminal according to the network protocol and played through the device's speaker to facilitate the user to receive information.

[0030] In a possible implementation, the converted instant messaging text message is sent to the user's communication terminal, or voice generation is performed on the converted instant messaging text message, and the converted language information is sent to the user's communication terminal. Step S350 further includes step S351, acquiring voice features of the first instant messaging message, and the language features include pitch, volume, speaking speed, timbre, intonation, and pause rhythm. Specifically, audio processing technology is used to obtain key acoustic features from the voice of instant messaging messages. First, the fast Fourier transform (FFT) is used to convert the time domain voice signal into a frequency domain signal. The energy distribution of the voice at different frequencies can be seen, which helps to extract information such as pitch, volume, and timbre. For example, pitch can be determined by analyzing the frequency of the fundamental frequency in the frequency domain, the volume is related to the energy of the signal, and the timbre is related to complex features such as the harmonic structure of the voice signal in the frequency domain. Mel-frequency cepstral coefficients (MFCC) are also used to further extract acoustic features that are more in line with human auditory perception characteristics, which can more effectively capture the essential characteristics of speech and provide rich information for subsequent speech simulation. Speech segmentation and annotation technology is used to identify pauses and sentence boundaries in speech to extract speech rate and pause rhythm. The speech segmentation algorithm can divide the continuous speech stream into segments according to the acoustic changes of the speech signal. For different speech units, such as words, phrases or sentences, annotation technology will mark the segmented units to clarify their boundaries and types. By analyzing the segmentation and annotation information, the number of speech units per unit time can be calculated, thereby obtaining the speech rate information. The position and duration of pauses can also be accurately identified, and then the pause rhythm can be determined. For example, in a longer speech, if there are obvious silent intervals, the duration and distribution of these intervals are important bases for the pause rhythm. Speech analysis tools, such as intonation analyzers, are used to identify and quantify intonation patterns. Intonation analyzers analyze the changing trends of speech signals in pitch. Different intonation patterns can convey different semantic and emotional information. For example, in declarative sentences, the intonation is usually relatively stable, while in interrogative sentences, the intonation often rises at the end of the sentence. Through quantitative analysis of intonation patterns, richer emotional and semantic expression information can be provided for subsequent speech simulation.

[0031] Step S352, using the pre-trained deep neural network model, voice simulation is performed according to the pitch, volume, speaking speed, timbre, intonation, pause rhythm and instant messaging text information to obtain a voice waveform and generate converted voice information.Specifically, choose a suitable pre-trained deep neural network model, such as a Transformer-based speech synthesis model, such as Tacotron or WaveNet. The model has shown excellent performance in the field of speech synthesis and can effectively handle the complex relationship between speech features and text information. The Tacotron model is good at converting text information into an intermediate spectral representation, while WaveNet can generate high-quality speech waveforms based on spectral information. According to the specific application scenarios and needs, you can choose one or combine the advantages of both to perform speech simulation, and use the extracted speech features and corresponding text information to fine-tune the selected model. It has unique voice features. By using a large amount of speech data of a specific speaker (including extracted speech features and corresponding text) to train the pre-trained model, the model can be better adapted to the speech pattern of the specific speaker. For example, if you want to simulate the voice of a specific person, you need to collect the voice sample of the person, extract the voice features and use them together with the corresponding text information to fine-tune the model. In this way, the model can more accurately reproduce the voice characteristics of the person in subsequent voice simulations. The text information of the instant messaging is input into the fine-tuned deep neural network model. The model generates the corresponding speech waveform based on the input text information and the previously extracted speech features. The model will comprehensively consider the semantics, linguistics and The model calculates the waveform value of the speech signal at each time point based on the structure of the word and various speech features. For example, for each word in the text, the model generates a suitable phoneme sequence and rhythmic pattern according to its part of speech, position in the sentence and corresponding speech features, and finally combines them into a continuous speech waveform. The generated speech is post-processed. For example, adding necessary pauses can make the speech more in line with human speaking habits and avoid sounding too mechanical. Adjusting the tone can make the speech more vivid according to the semantics and emotional color of the text. Properly adjusting the volume can highlight the key content or create a different emotional atmosphere. Through post-processing operations, the generated speech can be made more natural, fluent, and more emotionally expressive. Subjective and objective evaluations are performed to ensure its authenticity and emotional expression. Subjective evaluation can be completed by asking human listeners to score and evaluate the generated speech, and collect their feedback on speech naturalness, emotional expression, intelligibility, etc. Objective evaluation can use some speech quality evaluation indicators, such as speech clarity, prosodic similarity, etc. for quantitative analysis, and further optimize the model based on the evaluation results, such as adjusting the model parameters, increasing training data, or improving post-processing algorithms, etc., to continuously improve the quality of speech simulation, make the generated speech information more in line with user expectations, closer to real human speech, realize speech simulation of instant messaging messages, and generate speech information that sounds natural and has real emotions.

[0032] In a possible implementation, the instant messaging text message is translated using the preset language system of the first user to obtain the converted instant messaging text message, and step S340 further includes step S341, constructing a semantic translation model, the semantic translation model includes a semantic recognition sub-model, an emotion recognition sub-model, a polysemous recognition and labeling sub-model and a sub-model fusion module, and the sub-model fusion module is used to recognize and fuse the output results of the semantic recognition sub-model, the emotion recognition sub-model and the polysemous recognition and labeling sub-model. Specifically, a semantic recognition sub-model is constructed, whose main function is to understand the basic meaning and content of instant messaging text messages. Natural language processing (NLP) technology based on deep learning is adopted, such as using BERT (Bidirectional Encoder Representation from Transformer) or GPT (Generative Pre-training) models. The model is pre-trained on large-scale text data and can learn rich language knowledge and semantic representations. Among them, the BERT model can simultaneously consider the contextual information of words in a sentence through a bidirectional attention mechanism, so as to more accurately understand the semantics of the text. When processing instant messaging text messages, the model will convert the input text into a vector representation, and output the understanding result of the text semantics through the calculation of the neural network. An emotion recognition sub-model is constructed to identify the emotional color in instant messaging text messages and judge whether it is positive, negative or neutral. A method based on an emotion dictionary can be adopted. The method based on the emotion dictionary is to match the words in the text with a pre-constructed emotion dictionary, and judge the emotional color of the entire sentence according to the emotional polarity of the words, so as to construct a polysemous recognition sub-model. The annotation sub-model is used to identify polysemous words and ambiguous expressions in instant messaging text messages and annotate their correct meanings. It combines word sense disambiguation technology and context analysis. For example, the WordNet vocabulary knowledge base is used, which contains the different meanings of words and the semantic relationships between them. The deep learning model is used for context-sensitive word sense recognition. When processing text, the model will analyze the context of the word, combine the information in the vocabulary knowledge base, determine the exact meaning of the polysemous word in the current context, and annotate it. The sub-model fusion module is constructed, and its function is to identify and fuse the output results of the semantic recognition sub-model, the emotion recognition sub-model, and the polysemous recognition and annotation sub-model. Technically, integrated learning technology is used, such as stacking or multi-model fusion strategy. The strategy can fully combine the advantages of different sub-models to make the final fusion result more accurate and reliable. For example, the stacking method will use the output of each sub-model as a new feature, and then integrate it through another learner, so as to comprehensively consider semantics, emotion and polysemous information and make a more comprehensive decision.

[0033] Step S342, respectively use the semantic recognition sub-model, emotion recognition sub-model, and polysemy recognition and annotation sub-model to recognize and output the text information of the instant messaging, and obtain semantic recognition results, emotion feature recognition results, and polysemy recognition and annotation results. Specifically, the text information of the instant messaging is input into the semantic recognition sub-model, and the model processes the text based on its internal deep learning algorithm and pre-trained parameters. For example, for the sentence "I like this interesting activity", the semantic recognition sub-model can recognize that "I" is the subject, "like" is the action, "activity" is the object, and "interesting" is the description of the activity, thereby understanding the basic semantic content of the sentence and outputting the corresponding semantic recognition results. The instant messaging text information is input into the emotion recognition sub-model, and the model processes it according to the emotion analysis technology it adopts. For a sentence like "I like this interesting activity", based on the emotion dictionary The method may identify positive sentiment words such as "like" and "interesting" and judge that the sentence has positive sentiment. If a deep learning model, such as LSTM, is used, it will consider the sequence information of the entire sentence and also conclude that the sentence has positive sentiment. It will output the sentiment feature recognition result and pass the instant messaging text information to the polysemous recognition and annotation sub-model. The model will analyze the possible polysemous words. For example, the word "meaning" in "this activity is very interesting" has multiple meanings, but in this context, combined with the context, the polysemous recognition and annotation sub-model can determine its meaning of "interesting", annotate it, and output the polysemous recognition and annotation result.

[0034] Step S343, using the sub-model fusion module, the semantic recognition results, the emotional feature recognition results, and the polysemous recognition and annotation results are fused and corrected to determine the converted instant messaging text information, wherein the fusion correction is to use the emotional feature recognition results and the polysemous recognition and annotation results to fuse and correct the semantic recognition results, identify the emotional semantic expression according to the emotional feature recognition results and the polysemous recognition and annotation results, and correct the semantic recognition results using the emotional semantic expression. Specifically, the sub-model fusion module receives the semantic recognition results, the emotional feature recognition results, and the polysemous recognition and annotation results, and uses the emotional feature recognition results and the polysemous recognition and annotation results to correct the semantic recognition results. For example, if the semantic recognition results simply translate the literal meaning of the word, but the emotional feature recognition results show that the text is positive, and the polysemous recognition and annotation results show that a polysemous word has a special meaning in the current context, then it is necessary to adjust the semantic recognition results according to the information, and identify the emotional semantic expression according to the emotional feature recognition results and the polysemous recognition and annotation results. For example, in a sentence expressing love, the translation of certain words should reflect this positive emotion, and the emotional semantic expression is used to correct the semantic recognition results. Correction is performed to ensure that the final converted instant messaging text information is not only semantically accurate, but also that the emotional color and contextual meaning can be properly conveyed, avoiding inaccurate translation problems caused by not considering emotions and polysemy. In order to achieve better performance of the semantic translation model, a large amount of labeled data is needed for training. The labeled data includes the text content of the instant messaging, the corresponding emotional labels (such as positive, negative, neutral) and polysemy annotation information. During the training process, cross-validation technology is implemented to divide the data into multiple subsets, and different subsets are used in turn as validation sets to evaluate the performance of the model on unseen data. The generalization ability and accuracy of the model are measured through performance evaluation indicators such as accuracy, recall rate, F1 value, etc. According to the evaluation results, if it is found that the model is insufficient in processing certain types of text or emotions and polysemy, the model is iterated and optimized, such as adjusting the parameters of the model, adding more training data or improving the structure of the model to improve the overall performance of the model.

[0035] In one possible implementation, the semantic recognition sub-model, the emotion recognition sub-model, and the polysemy recognition and labeling sub-model are used to recognize and output the text information of the instant messaging, respectively, to obtain semantic recognition results, emotion feature recognition results, and polysemy recognition and labeling results. Step S342 further includes step S3421, constructing an emotion memory library for storing emotion-related words recognized in the conversation. Specifically, firstly, a suitable database or data structure is created as an emotional memory bank to facilitate the storage and retrieval of emotional related words. For example, a relational database can be used to store words in a table form, or a specific data structure such as a hash table can be used to improve the retrieval efficiency. The scope of emotional related words is clarified, including emotional words (such as "happy", "sad", "angry", etc.), emotional expression phrases (such as "elated", "dejected", "furious", etc.) and emotional tendency words (such as "good" tends to be positive emotions, and "bad" tends to be negative emotions). At the beginning, some common emotional words and expressions are filled into the emotional memory bank. The initial content is based on extensive language knowledge and understanding of general emotional expressions, such as "happy" and "happy" in the positive emotional category, and "sad" and "painful" in the negative emotional category. These initialized words provide a basis for subsequent emotional analysis and association extraction.

[0036] Step S3422, based on the emotional memory library, the text information of the instant messaging is identified, the emotional association words of the target information are extracted, and the first-order interactive emotional association words are obtained. Specifically, after receiving the text information of the instant messaging, the emotional analysis is performed on it, and the text information is decomposed into words and phrases using natural language processing technology, such as lexical analysis, syntactic analysis, etc., and their grammatical roles in the sentence are analyzed. In the process of analyzing the text information, those emotional association words that directly express emotional colors are identified from the target information. For example, in the sentence "I am really happy today because my test results are great", "happy" and "great" are words that directly reflect positive emotions. Once identified, the emotional association words are extracted, and the extracted emotional association words are stored in the emotional memory library, further enriching the content of the emotional memory library, so as to perform more accurate emotional association analysis in subsequent larger-scale conversations.

[0037] Step S3423, based on the first-order interactive emotional association words, the associated dialogue capture and extraction are performed in the emotional memory library to obtain the second-order interactive emotional association words. Specifically, based on the first-order interactive emotional association words extracted from the target information, the emotional memory library is searched and associated with analysis. For example, if the first-order interactive emotional association word is "happy", other words or phrases related to "happy" are searched in the emotional memory library. In the process of associated search, other emotional words related to the first-order emotional association words are identified and extracted. The words may not directly express emotions, but indirectly reflect emotional colors or emotional tendencies. For example, words such as "celebration" and "cheer" related to "happy" do not directly describe the state of happiness, but are closely related to the emotion of happiness. These words are second-order interactive emotional association words.

[0038] Step S3424, using the first-order interactive emotion-related words and the second-order interactive emotion-related words to aggregate emotion features, and outputting the emotion feature recognition result based on the aggregated emotion features. Specifically, the first-order interactive emotion-related words and the second-order interactive emotion-related words are combined to aggregate emotion features. The aggregation process is implemented in a variety of ways, such as calculating the frequency of emotion words. If "happy" and its related second-order words appear frequently in a conversation, it means that positive emotions dominate in this conversation. The intensity of the emotion tendency can be analyzed, and different weights can be assigned according to the intensity of the emotion expressed by the words. For example, "ecstasy" has a higher positive emotion intensity than "pleased". The contextual relationship of emotion words is considered. For example, under a specific topic, the meaning and emotion tendency of some emotion words may change. The emotion features can be more accurately aggregated by analyzing the context. Through the above aggregation method, a more comprehensive and multi-dimensional emotion feature description is obtained. The description is not limited to the emotion expressed by a single word, but comprehensively considers direct and indirect emotion expressions, as well as their distribution and mutual relationship in the conversation. , emotion recognition is performed based on the aggregated emotion features, and the emotion feature recognition results are output. The results can reflect the emotional tendency (positive, negative or neutral) in the conversation and the approximate range of emotion intensity, providing an important basis for further understanding the conversation content and user emotions. As the instant messaging conversation continues, newly emerging emotion-related words and new relationships between them are continuously added to the emotion memory bank. For example, when new popular online emotion expression words appear, they are promptly included in the emotion memory bank so that the emotion memory bank can adapt to the ever-changing language environment and user expression habits. The emotion memory bank is regularly maintained to check the emotion-related words in the bank and remove those words that are no longer relevant over time or with language changes. At the same time, the structure and content of the emotion memory bank are optimized, such as adjusting the storage method to improve retrieval efficiency, to ensure that the emotion memory bank always maintains efficient and accurate support for emotion analysis.

[0039] In a possible implementation, the first-order interactive emotion association words and the second-order interactive emotion association words are used to perform emotion feature aggregation, and the emotion feature recognition result is output based on the aggregated emotion features. Step S3424 further includes step S34241, performing emotion category and emotion intensity recognition based on the second-order interactive emotion association words, and combining the number of occurrences of the emotion association words and context information to obtain the second-order emotion features. Specifically, the second-order interactive emotional associations are analyzed, and the emotional categories are determined based on the emotional tendencies contained in the words themselves. For example, words such as "relieved" and "joyed" can be classified as positive emotional categories; "sorrow" and "sorrow" belong to negative emotional categories. For some words with ambiguous emotional tendencies, they are judged based on the information in the emotional memory bank and their general emotional orientation in the language. For example, the word "surprise" is neutral, positive or negative in most cases, and needs to be further determined based on the specific context. The emotional intensity expressed by the second-order interactive emotional associations is evaluated. Different emotional words have different intensity levels. For example, the positive emotional intensity of "ecstasy" is much higher than that of "happy"; the negative emotional intensity of "heartbroken" is also much higher than that of "a little sad." This can be determined by a pre-set sentiment intensity dictionary or a sentiment intensity value of a word obtained based on statistical analysis of a large amount of corpus. The number of occurrences of second-order interactive sentiment associated words and other sentiment associated words closely related to them in instant messaging text messages is counted. Sentiment associated words with higher frequencies can, to a certain extent, reflect the importance of the emotion in this message. For example, if "happy" and its related words appear frequently in a conversation, it means that positive emotions occupy a more prominent position in it. The context of the second-order interactive sentiment associated words is analyzed. The context can affect the meaning and intensity of sentiment vocabulary. For example, "this result is really unexpected". The emotion of "unexpected" alone is not clear, but if the previous text mentions "the good result that has been expected has appeared", then "unexpected" tends to be a positive emotion; if the previous text is "the plan failed", then "unexpected" tends to be a negative emotion. Combining these factors to obtain second-order sentiment characteristics is a more comprehensive and detailed description of the emotions contained in second-order interactive sentiment associated words.

[0040] Step S34242, using the second-order interactive emotional association words and the first-order interactive emotional association words to perform correlation analysis, and determine the correlation coefficient. Specifically, the correlation analysis of the second-order interactive emotional association words and the first-order interactive emotional association words can be performed by calculating the co-occurrence frequency of words, based on the semantic similarity algorithm, etc. If the first-order interactive emotional association word is "happy", and the second-order interactive emotional association words include "laughter" and "celebration", by analyzing a large amount of text data, it is found that they often co-occur in similar contexts, indicating that they have a strong correlation. The correlation coefficient is determined according to the result of the correlation analysis. The correlation coefficient is a quantitative value that indicates the closeness of the relationship between the first-order interactive emotional association words and the second-order interactive emotional association words. For example, after calculation and analysis, the correlation coefficients of "happy" and "laughter" may be high, while the correlation coefficients with some less relevant words are low. This coefficient provides a weight basis for the subsequent aggregation of emotional features.

[0041] Step S34243, using the correlation coefficient as the aggregation weight, the second-order emotional features are aggregated to obtain the emotional aggregation features, including emotional categories and emotional intensity. Specifically, the determined correlation coefficient is used as the aggregation weight to perform weighted aggregation on the second-order emotional features. For each second-order emotional feature, a corresponding weighting process is performed according to its correlation coefficient with the first-order interactive emotional associated words. For example, if a second-order emotional feature has a high correlation coefficient, it means that it accounts for a large proportion of the overall emotional expression, and its weight is large during the aggregation process. Through weighted aggregation, the emotional aggregation features are obtained. The features not only include the emotional categories, that is, the positive, negative or neutral emotional tendencies of the final comprehensive judgment, but also include the quantitative indicator of emotional intensity, which reflects the intensity of the emotions expressed by the entire instant messaging text information.

[0042] Step S34244, obtaining the emotion feature recognition result according to the emotion category and its emotion intensity. Specifically, according to the aggregated emotion category and emotion intensity, finally determining the emotion feature recognition result, the result can accurately reflect the emotion information contained in the instant messaging text information. For example, if the emotion category is positive and the emotion intensity is high, then the emotion feature recognition result can be expressed as "the user is in a strong positive emotion state", which provides an important basis for further understanding the user's emotions, attitudes and subsequent interaction strategy formulation.

[0043] Step S400, the first instant conversion language information is sent to the communication terminal of the first user, and the second instant messaging message sent by the first user is converted into a language to obtain the second instant conversion language information, which is the language information after the second instant messaging message is converted from the preset language system of the first user to the preset language system of the second user. Specifically, after obtaining the first instant conversion language information, the system encapsulates it, adds header information such as the source of the message, the target user, and the message type, selects a transmission path according to the network environment and communication protocol (through switches and routers in the local area network, through the Internet routing protocol in the wide area network, and forwards with the help of DNS querying the target server IP address), and sends the information to the communication terminal of the first user, and the instant messaging application of the terminal decapsulates it and displays it on the interface. At the same time, the system receives the second instant messaging message sent by the first user, checks the preset language systems of the first user and the second user (obtained from the system database), and if the message is text information, performs grammatical analysis and semantic understanding on it and performs vocabulary conversion and grammatical reconstruction according to the target language system (such as converting the first user's English message into the second user's French message, such as changing "I love you." to "Je t'aime."). If it is voice information, it is first recognized as text and then processed according to the text information conversion process. If necessary, it is converted into voice information of the target language system to obtain the second instant conversion language information.

[0044] Step S500, the second instant conversion language information is sent to the communication terminal of the second user to establish instant communication between the first user and the second user. Specifically, after obtaining the second instant conversion language information, the system adds metadata such as the source of the message, the target user, and the message type to encapsulate it, and then plans the transmission path according to the network topology and communication protocol (in the local area network, the path is determined by using switches and routers according to the internal routing table, and in the wide area network, the path is determined by querying the target IP address through DNS according to the Internet routing protocol such as BGP), and the information is sent to the communication terminal of the second user through network transmission. The instant messaging application of the terminal decapsulates and extracts the content. When the second user can receive and view the message that conforms to the preset language system, the instant communication between the first user and the second user is established.

[0045] The embodiment of the present application obtains the preset language system of the first user and the second user, and determines whether their language systems are consistent when the two establish communication. If they are different, the message sent by the second user is converted into the language system of the first user and sent to the terminal of the first user, and the message sent by the first user is converted into the language system of the second user and sent to the terminal of the second user, thereby realizing language synchronization of instant messaging and achieving the technical effect of realizing efficient and accurate cross-language instant messaging.

[0046] In the above, refer to Figure 1 A method for synchronously processing instant messaging messages according to an embodiment of the present invention is described in detail. Figure 2 A system for synchronously processing instant messaging messages according to an embodiment of the present invention is described.

[0047] According to an embodiment of the present invention, a synchronous processing system for instant messaging messages is used to solve the technical problem that the existing instant messaging tools are inefficient and inaccurate in real-time conversion of multiple languages, and achieves the technical effect of realizing efficient and accurate cross-language instant messaging. A synchronous processing system for instant messaging messages includes: a preset language system acquisition module 10, a preset language system judgment module 20, a first instant conversion language information acquisition module 30, a language conversion module 40, and an instant messaging establishment module 50.

[0048] The preset language system acquisition module 10 is used to obtain the preset language system of the first user and the preset language system of the second user.

[0049] The preset language system determination module 20 is used to determine whether the preset language system of a first user is consistent with the preset language system of a second user when a first user establishes instant messaging with a second user.

[0050] The first instant conversion language information acquisition module 30 is used to convert the language of the first instant messaging message sent by the second user according to the preset language system of the first user if the preset language system of the first user is different from the preset language system of the second user, so as to obtain first instant conversion language information. The first instant conversion language information is the language information after the first instant messaging message is converted from the preset language system of the second user to the preset language system of the first user.

[0051] The language conversion module 40 is used to send the first instant conversion language information to the communication terminal of the first user, and at the same time convert the language of the second instant messaging message sent by the first user to obtain the second instant conversion language information, where the second instant conversion language information is the language information after the second instant messaging message is converted from the preset language system of the first user to the preset language system of the second user.

[0052] The instant messaging establishment module 50 is used to send the second instant messaging conversion language information to the communication terminal of the second user to establish the instant messaging between the first user and the second user.

[0053] The specific configuration of the preset language system acquisition module 10 will be described in detail below. As described above, the preset language system of the first user and the preset language system of the second user are obtained, and the preset language system acquisition module 10 further includes: a preset language system identification unit, which is used to identify the preset language system of each user when there are more than two users participating in establishing the instant communication; a multi-conversion channel establishment unit, which is used to establish multiple conversion channels based on the preset language system of each user with a single participating user as the target, and the multiple conversion channels include information conversion channels of each preset language system of the target user and other communication users; a language system conversion unit, which is used to obtain instant messaging information sent by non-target users in the instant messaging, perform language system conversion through the multiple conversion channels, generate instant messaging information of the target language system, and send it to the communication terminal of the target user.

[0054] Among them, the instant messaging information sent by non-target users in the instant messaging is obtained, the language system is converted through the multiple conversion channels respectively, and the instant messaging information of the target language system is generated and sent to the communication terminal of the target user. The language system conversion unit further includes: an instant messaging information composition subunit, and the instant messaging information composition subunit is used for the instant messaging information including: text information and voice information.

[0055] The specific configuration of the first instant conversion language information acquisition module 30 will be described in detail below. As described above, if the preset language system of the first user is different from the preset language system of the second user, the language of the first instant messaging message sent by the second user is converted according to the preset language system of the first user to obtain the first instant conversion language information, the first instant conversion language information is the language information after the first instant messaging message is converted from the preset language system of the second user to the preset language system of the first user, the first instant conversion language information acquisition module 30 further includes: a semantic information acquisition unit, the semantic information acquisition unit is used to obtain semantic information according to the preset language system of the first instant messaging message and its corresponding second user when the instant messaging message is text information; a text conversion unit, the text conversion unit is used to convert the semantic information into text using the preset language system of the first user to obtain the instant conversion language information after the language system conversion.

[0056] Among them, the first instant conversion language information acquisition module 30 further includes: a text extraction unit, which is used to extract text according to the first instant messaging message and the preset language system of the second user when the instant messaging message is language information, so as to obtain the text information of the instant messaging; a text information translation unit, which is used to translate the instant messaging text information using the preset language system of the first user to obtain the converted instant messaging text information; a voice generation unit, which is used to send the converted instant messaging text information to the user's communication terminal, or to generate voice for the converted instant messaging text information and send the converted language information to the user's communication terminal.

[0057] Among them, the converted instant messaging text message is sent to the user's communication terminal, or the converted instant messaging text message is subjected to voice generation, and the converted language information is sent to the user's communication terminal. The voice generation unit further includes: a voice feature acquisition subunit, the voice feature acquisition subunit is used to acquire the voice features of the first instant messaging message, and the language features include pitch, volume, speaking speed, timbre, intonation, and pause rhythm; a voice waveform acquisition subunit, the voice waveform acquisition subunit is used to use a pre-trained deep neural network model to perform voice simulation according to the pitch, volume, speaking speed, timbre, intonation, pause rhythm and instant messaging text information to obtain a voice waveform, and generate converted voice information.

[0058] The instant messaging text message is translated using the preset language system of the first user to obtain the converted instant messaging text message. The text message translation unit further includes: a semantic translation model construction subunit, the semantic translation model construction subunit is used to construct a semantic translation model, the semantic translation model includes a semantic recognition submodel, an emotion recognition submodel, a polysemous recognition and annotation submodel and a submodel fusion module, the submodel fusion module is used to recognize and fuse the output results of the semantic recognition submodel, the emotion recognition submodel and the polysemous recognition and annotation submodel; a text information recognition output subunit, the text information recognition output subunit is used to respectively use the semantic recognition submodel, the emotion recognition submodel and the polysemous recognition and annotation submodel to generate a text message. The identification sub-model and the polysemous recognition and labeling sub-model recognize and output the text information of the instant messaging to obtain semantic recognition results, emotional feature recognition results, and polysemous recognition and labeling results; a fusion and correction sub-unit is used to use the sub-model fusion module to fuse and correct the semantic recognition results, emotional feature recognition results, and polysemous recognition and labeling results to determine the converted instant messaging text information, wherein the fusion and correction is to use the emotional feature recognition results and the polysemous recognition and labeling results to fuse and correct the semantic recognition results, identify emotional semantic expressions based on the emotional feature recognition results and the polysemous recognition and labeling results, and correct the semantic recognition results using the emotional semantic expressions.

[0059] Among them, the semantic recognition sub-model, the emotion recognition sub-model, and the polysemy recognition and annotation sub-model are respectively used to recognize and output the text information of the instant messaging to obtain semantic recognition results, emotion feature recognition results, and polysemy recognition and annotation results. The text information recognition and output sub-unit further includes: an emotion memory library construction micro-unit, the emotion memory library construction micro-unit is used to construct an emotion memory library for storing emotion-related words recognized in the conversation; an interactive emotion-related word acquisition micro-unit, the interactive emotion-related word acquisition micro-unit is used to recognize the text information of the instant messaging based on the emotion memory library, extract the emotion-related words of the target information, and obtain first-order interactive emotion-related words; an associated conversation capture and extraction micro-unit, the associated conversation capture and extraction micro-unit is used to perform associated conversation capture and extraction in the emotion memory library based on the first-order interactive emotion-related words to obtain second-order interactive emotion-related words; an emotion feature aggregation micro-unit, the emotion feature aggregation micro-unit is used to aggregate emotion features using the first-order interactive emotion-related words and the second-order interactive emotion-related words, and output the emotion feature recognition result based on the aggregated emotion features.

[0060] Among them, the first-order interactive emotional association words and the second-order interactive emotional association words are used to aggregate emotional features, and the emotional feature recognition result is output based on the aggregated emotional features. The emotional feature aggregation micro-unit further includes: a second-order emotional feature acquisition unit, the second-order emotional feature acquisition unit is used to identify the emotional category and emotional intensity according to the second-order interactive emotional association words, and obtain the second-order emotional features in combination with the number of occurrences of the emotional association words and context information; a correlation coefficient determination unit, the correlation coefficient determination unit is used to use the second-order interactive emotional association words and the first-order interactive emotional association words to perform correlation analysis and determine the correlation coefficient; a feature aggregation unit, the feature aggregation unit is used to use the correlation coefficient as the aggregation weight to aggregate the second-order emotional features to obtain the emotional aggregation features, including the emotional category and the emotional intensity; an emotional feature recognition result acquisition unit, the emotional feature recognition result acquisition unit is used to obtain the emotional feature recognition result according to the emotional category and its emotional intensity.

[0061] The instant messaging message synchronization processing system provided by the embodiment of the present invention can execute the instant messaging message synchronization processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0062] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0063] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for synchronously processing instant messaging messages, characterized in that: The method comprises: Obtaining a preset language system of a first user and a preset language system of a second user; When a first user establishes instant messaging with a second user, determining whether a preset language system of the first user is consistent with a preset language system of the second user; If the preset language system of the first user is different from the preset language system of the second user, performing language conversion on the first instant messaging message sent by the second user according to the preset language system of the first user to obtain first instant conversion language information, where the first instant conversion language information is language information after the first instant messaging message is converted from the preset language system of the second user to the preset language system of the first user; The first instant conversion language information is sent to the communication terminal of the first user, and the second instant messaging message sent by the first user is converted into a language to obtain second instant conversion language information, where the second instant conversion language information is language information after the second instant messaging message is converted from a preset language system of the first user into a preset language system of the second user; The second instant conversion language information is sent to a communication terminal of a second user to establish instant communication between the first user and the second user.

2. The method for synchronously processing instant messaging messages according to claim 1, characterized in that: The method further comprises: When there are more than two users participating in establishing the instant messaging, identifying the preset language system of each user; Taking a single participating user as a target, establishing multiple conversion channels according to the preset language system of each user, wherein the multiple conversion channels include information conversion channels of the preset language systems of the target user and other communicating users; Instant messaging information sent by non-target users in instant messaging is acquired, language system conversion is performed through the multiple conversion channels respectively, instant messaging information in the target language system is generated, and sent to the communication terminal of the target user.

3. The method for synchronously processing instant messaging messages according to claim 2, characterized in that: The instant messaging information includes: text information and voice information.

4. The method for synchronously processing instant messaging messages according to claim 3, characterized in that: The first instant messaging message sent by the second user is converted into a language according to a preset language system of the first user, including: When the instant messaging message is a text message, obtaining semantic information according to the preset language system of the first instant messaging message and the corresponding second user; The semantic information is converted into text using the preset language system of the first user to obtain instant converted language information after the language system conversion.

5. The method for synchronously processing instant messaging messages according to claim 3, characterized in that: The method further includes: converting the language of the first instant messaging message sent by the second user according to the preset language system of the first user; When the instant messaging message is language information, text extraction is performed according to the first instant messaging message and the preset language system of the second user to obtain the text information of the instant messaging; Translating the instant messaging text message using the preset language system of the first user to obtain converted instant messaging text message; The converted instant messaging text message is sent to the user's communication terminal, or voice is generated for the converted instant messaging text message, and the converted language information is sent to the user's communication terminal.

6. The method for synchronously processing instant messaging messages according to claim 5, characterized in that: Generating speech from the converted instant messaging text message includes: Acquire voice features of the first instant messaging message, wherein the voice features include pitch, volume, speaking speed, timbre, intonation, and pause rhythm; Using a pre-trained deep neural network model, voice simulation is performed according to the pitch, volume, speaking speed, timbre, intonation, pause rhythm and instant messaging text information to obtain a voice waveform and generate converted voice information.

7. The method for synchronously processing instant messaging messages according to claim 6, characterized in that: The instant messaging text message is translated using the preset language system of the first user to obtain the converted instant messaging text message, including: Constructing a semantic translation model, the semantic translation model comprising a semantic recognition sub-model, an emotion recognition sub-model, a polysemous recognition and annotation sub-model, and a sub-model fusion module, the sub-model fusion module being used to recognize and fuse the output results of the semantic recognition sub-model, the emotion recognition sub-model, and the polysemous recognition and annotation sub-model; Respectively using the semantic recognition sub-model, the emotion recognition sub-model, and the polysemy recognition and annotation sub-model to recognize and output the text information of the instant messaging to obtain a semantic recognition result, an emotion feature recognition result, and a polysemy recognition and annotation result; The sub-model fusion module is used to fuse and correct the semantic recognition results, emotional feature recognition results, and polysemous recognition and annotation results to determine the converted instant messaging text information, wherein the fusion correction is to fuse and correct the semantic recognition results using the emotional feature recognition results and polysemous recognition and annotation results, identify emotional semantic expressions based on the emotional feature recognition results and polysemous recognition and annotation results, and correct the semantic recognition results using the emotional semantic expressions.

8. The method for synchronously processing instant messaging messages according to claim 7, characterized in that: The emotion recognition sub-model is used to recognize and output the text information of the instant messaging, including: Build an emotional memory library to store emotion-related words identified in the conversation; Based on the emotion memory library, the text information of the instant messaging is identified, the emotion-related words of the target information are extracted, and the first-order interactive emotion-related words are obtained; Based on the first-order interactive emotion-related words, relevant conversations are captured and extracted in the emotion memory library to obtain second-order interactive emotion-related words; The first-order interactive emotion-related words and the second-order interactive emotion-related words are used to aggregate emotion features, and the emotion feature recognition result is output based on the aggregated emotion features.

9. The method for synchronously processing instant messaging messages according to claim 8, characterized in that: The first-order interactive emotion-related words and the second-order interactive emotion-related words are used to aggregate emotion features, and the emotion feature recognition result is output based on the aggregated emotion features, including: Identify the emotion category and emotion intensity according to the second-order interactive emotion-related words, and obtain the second-order emotion features by combining the number of occurrences of the emotion-related words and context information; Performing correlation analysis using the second-order interactive sentiment associated words and the first-order interactive sentiment associated words to determine a correlation coefficient; The second-order sentiment features are aggregated using the correlation coefficient as an aggregation weight to obtain the sentiment aggregation features, including sentiment category and sentiment intensity; The emotion feature recognition result is obtained according to the emotion category and its emotion intensity.

10. A system for synchronously processing instant messaging messages, characterized in that: The system is used to implement the method for synchronously processing instant messaging messages according to any one of claims 1 to 9, and the system comprises: A preset language system acquisition module, the preset language system acquisition module is used to obtain a preset language system of the first user and a preset language system of the second user; A preset language system determination module, which is used to determine whether the preset language system of the first user is consistent with the preset language system of the second user when the first user establishes instant messaging with the second user; a first instant conversion language information acquisition module, the first instant conversion language information acquisition module being used to, if the preset language system of the first user is different from the preset language system of the second user, perform language conversion on the first instant messaging message sent by the second user according to the preset language system of the first user to obtain first instant conversion language information, the first instant conversion language information being language information after the first instant messaging message is converted from the preset language system of the second user to the preset language system of the first user; a language conversion module, the language conversion module being used to send the first instant conversion language information to the communication terminal of the first user, and simultaneously convert the language of the second instant messaging message sent by the first user to obtain the second instant conversion language information, wherein the second instant conversion language information is the language information after the second instant messaging message is converted from the preset language system of the first user to the preset language system of the second user; An instant messaging establishment module is used to send the second instant messaging conversion language information to a communication terminal of a second user to establish instant messaging between the first user and the second user.

Images