Voice processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a voice processing method and device, electronic equipment and a storage medium, and belongs to the technical field of electronic equipment. The method comprises the steps of displaying voice subtitles according to voices of participants under the condition that the voices of the participants are received; wherein the voice subtitles comprise the voice translation text, or the voice text and the voice translation text.

Description

Technical Field

[0001] The present application belongs to the technical field of electronic equipment, and specifically relates to a speech processing method, device, electronic equipment and storage medium. Background Art

[0002] With the popularization of mobile Internet and smart devices, users have an increasing demand for functions such as voice interaction, language translation, and real-time information processing in their daily work and life.

[0003] Currently, in a conference scenario, if there are participants who speak different languages, each participant needs to rely on a translator or a translation machine to translate sentence by sentence, and the communication efficiency between the participants is low. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a speech processing method, device, electronic device and storage medium to improve the communication efficiency between conference participants who speak different languages.

[0005] In a first aspect, an embodiment of the present application provides a speech processing method, the method comprising:

[0006] When receiving the voice of a conference member, displaying voice subtitles according to the voice of the conference member;

[0007] The voice subtitles include voice translation text, or voice text and voice translation text.

[0008] In a second aspect, an embodiment of the present application provides a speech processing device, the device comprising:

[0009] A display module, configured to display voice subtitles according to the voice of a conference member when receiving the voice of the conference member;

[0010] The voice subtitles include voice translation text, or voice text and voice translation text.

[0011] In a third aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0012] In a fourth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0013] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0014] In an embodiment of the present application, when the conference members are conference members who speak different languages ​​and when the voice of the conference member is received, the translated text of the conference member's voice can be intuitively displayed, or the translated text and voice text of the conference member's voice can be displayed, so that the user can intuitively view the translated text of the conference member's voice, or the translated text and voice text of the conference member's voice, and then communicate with the conference members based on the translated text of the conference member's voice, or the translated text and voice text of the conference member's voice, thereby improving the communication efficiency between conference members who speak different languages. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flowchart of a speech recognition method provided by some embodiments of the present application;

[0016] Figure 2 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0017] Figure 3 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0018] Figure 4 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0019] Figure 5 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0020] Figure 6 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0021] Figure 7 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0022] Figure 8 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0023] Fig. 9 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0024] Fig.10 is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0025] Fig.11is a schematic diagram of a voice recognition function interface provided by some embodiments of the present application;

[0026] Fig.12 is a schematic diagram of the structure of a speech processing device shown in some embodiments of the present application;

[0027] Fig.13 is a schematic diagram of the structure of an electronic device shown in some embodiments of the present application;

[0028] Fig.14 It is a schematic diagram of the hardware structure of an electronic device shown in some embodiments of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0030] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or N. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0031] As described in the background technology section, in the existing solution, in a conference scenario, if there are participants speaking different languages, the communication efficiency between the participants is low. In order to solve the above problem, the embodiments of the present application provide a voice processing method, device, electronic device and storage medium. When the participants are participants using different languages ​​and when the voice of the participants is received, the translated text of the participant's voice can be intuitively displayed, or the translated text and voice text of the participant's voice can be displayed, so that the user can intuitively view the translated text of the participant's voice, or the translated text and voice text of the participant's voice, and then communicate with the participants based on the translated text of the participant's voice, or the translated text and voice text of the participant's voice, thereby improving the communication efficiency between participants speaking different languages.

[0032] The technical solution of the embodiment of the present application can be applied to the scene of multi-person multi-language communication to recognize and translate each person's voice information and display voice subtitles. The embodiment of the present application takes the multi-person multi-language conference communication scene as an example for explanation, but the solution of the embodiment of the present application is not limited to the multi-person multi-language conference communication scene, and can also be distance education and multinational online classrooms, multi-language customer service, international conferences and business negotiations, multi-language project collaboration meetings, multi-language news releases and media interviews, etc.

[0033] The speech processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0034] Figure 1 It is a flow chart of a speech processing method provided in an embodiment of the present application. The executor of the speech processing method may be an electronic device, which may specifically be an electronic device held by a user. The electronic device may be, but is not limited to, a personal computer (PC), a smart phone, a tablet computer or a personal digital assistant (PDA), etc.

[0035] The voice processing method provided in the embodiment of the present application may include: when receiving the voice of a conference member, displaying voice subtitles according to the voice of the conference member; wherein the voice subtitles include voice translation text, or voice text and voice translation text.

[0036] The conference participants may be members participating in the conference, and may specifically include users and other members.

[0037] The voice subtitles may be subtitles corresponding to the voices of the conference participants, and the voice subtitles may include voice translation text, or voice text and voice translation text.

[0038] The above-mentioned speech translation text may be text information after translating the speech of the conference member, and specifically may be text information after translating the speech of the conference member in a certain language into another language desired by the user. For example, the language used by conference member A is English, but the user's idiomatic language is Chinese, then the English speech of conference member A may be converted into a Chinese speech translation text.

[0039] The voice text may be text converted from the voice of the conference participants.

[0040] It should be noted that, in step 110, there may be multiple conference participants, and the voices of the conference participants may be in multiple languages.

[0041] In some embodiments of the present application, the voices of the conference members may include the first voice of the first object and the second voice of the second object, where the first object and the second object may be the conference members respectively, the above-mentioned first voice may be the voice of the first object, the second voice may be the voice of the second object, and the language of the first voice is the first language, and the language of the second voice is the second language, where the first language and the second language are two different languages, for example, the first language is English and the second language is Chinese.

[0042] During the process of receiving the voices of the participating members, the voices of each participating member can be recognized separately. Specifically, the voices of each participating member can be converted into text to obtain voice text corresponding to the voice of each participating member. The voices of each participating member can also be translated into another language to obtain voice translation text corresponding to the voice of each participating member, and the voice translation text and / or voice text corresponding to the voice of each participating member can be displayed.

[0043] In one example, reference Figure 2 For example, if the language corresponding to the voice translation text is Chinese, the voice message of participant A in the meeting is "When does the meeting start?", and the voice message of participant B in the meeting is "How's the project progress?", then the voice message of participant A "When does the meeting start?" can be translated into Chinese "When does the meeting start?", and the voice message of participant B "How's the project progress?" can be translated into Chinese "How is the project progress?" and displayed.

[0044] In some embodiments of the present application, the voice of the conference member received may be the voice of the conference member received in real time through the voice receiving device, or may be the voice of the conference member played from other audio devices received by the voice receiving device in the conference scene, such as the voice of the conference member received from a recording device, etc. The specific voice acquisition method is not limited here, and any method that can acquire the voice of the conference member in the conference scene belongs to the protection scope of the embodiments of the present application.

[0045] It should be noted that the voice of the conference member is received when the conference member enters the conference scene. The conference member entering the conference scene may be the conference member entering the conference scene through the conference link. After the user enters the conference scene, the electronic device will automatically start the voice recognition function and display the voice recognition function interface. The voice recognition function interface can be superimposed on the conference interface in the form of a floating window. Specifically, the voice recognition function interface can be an interface designed based on artificial intelligence (AI) technology, that is, the voice recognition function interface can be an AI interactive interface.

[0046] The above-mentioned speech recognition function can be a function developed based on AI technology, which can recognize speech in multiple languages ​​at the same time and convert it into corresponding speech text and / or speech translation text. Compared with the existing speech recognition function with a single function, the speech recognition function provided in the embodiment of the present application can recognize speech in multiple languages ​​at the same time, meeting the diverse needs of recognizing speech in different languages ​​in a multilingual environment.

[0047] In one example, reference Figure 3 After the user enters the meeting, the electronic device automatically starts the voice recognition function and displays the AI ​​interaction interface 31, which is superimposed on the meeting interface in the form of a floating window.

[0048] In some embodiments of the present application, displaying voice subtitles according to the voice of the conference participants may specifically include:

[0049] Display the voice subtitles of the participants on the artificial intelligence (AI) interactive interface.

[0050] In some embodiments of the present application, after the AI ​​interaction interface is displayed, the voice subtitles of the conference participants can be displayed in the AI ​​interaction interface, such as Figure 2 shown.

[0051] In an embodiment of the present application, by superimposing the voice subtitles of the participating members in the AI ​​interactive interface displayed in the conference interface, the user does not need to exit the conference interface and switch to the AI ​​interactive interface to view the voice subtitles of the participating members. That is, the user can intuitively view the voice subtitles of the participating members without switching interfaces back and forth, thereby improving the viewing efficiency of the voice subtitles of the participating members.

[0052] In some embodiments of the present application, in order to further improve the efficiency of conference communication, the display of voice subtitles according to the voices of the conference participants may specifically include:

[0053] Determine the object identification of the conference member according to the voice of the conference member;

[0054] Displays the object IDs and voice subtitles of conference participants.

[0055] The object identifier may be an identifier used to uniquely identify a conference participant.

[0056] In some embodiments of the present application, the timbre characteristics in the speech of the conference member can be analyzed to identify which conference member the received speech belongs to, and the object identification of the conference member can be obtained. In this way, when the speech subtitles of the conference member are displayed, the object identification of the conference member can be displayed together. Specifically, the speech subtitles can be located in the associated area of ​​the object identification of the conference member.

[0057] The above-mentioned associated area may be a certain area preset in the object identifier of the conference member, for example, it may be an area 2 mm below the object identifier of the conference member.

[0058] Continuing with the above examples and Figure 2 After translating the voice of participant A, “When does the meeting start?” into Chinese, “When does the meeting start?”, and translating the voice of participant B, “How's the project progress?” into Chinese, “How is the project progress?”, they can be displayed. In addition, the timbre features in the voices of participant A and participant B can be analyzed to determine that the voice “When does the meeting start?” and the voice “How's the project progress?” come from different participants, respectively. Then, the object identifiers of the participant corresponding to the voice “When does the meeting start?” and the voice “How's the project progress?” can be displayed respectively. For example, the object identifier of the voice “When does the meeting start?” is object A, and the object identifier of the voice “How's the project progress?” is object B.

[0059] It should be noted that, when identifying the conference participants corresponding to the voices, only different conference participants corresponding to different voices can be identified, but it is not possible to determine which voice is spoken by which person. Figure 2 When the object identifier is displayed in the , it is only to distinguish that different voices come from different participants, but it cannot determine who the specific participant is. Figure 2As shown, it can only be recognized that the voice "When does the meeting start?" and the voice "How's the project progress?" come from different meeting participants, and different object identifiers are assigned to the voice "When does the meeting start?" and the voice "How's the project progress?", but it is not certain who said the voice "When does the meeting start?", for example, whether it was said by Zhang San or Li Si, and whether the voice "How's the project progress?" was said by Zhang San or Li Si.

[0060] In another embodiment, when displaying the object identification and voice subtitles of the conference participants, if the user also speaks, but speaks in an idiomatic language, the user's voice can also be translated into the language used by most other objects, that is, voice information can be displayed in two languages ​​in the electronic device.

[0061] refer to Figure 2 For example, when English is the language used by most participants, after participant A asks “When does the meeting start?”, the user replies in Chinese “The meeting starts at 10 AM”. The Chinese reply “The meeting starts at 10 AM” and the corresponding English version of the Chinese reply “The meeting starts at 10 AM” can also be displayed on the electronic device, so that the user can intuitively check whether the text translated from the Chinese voice into English is correct.

[0062] In an embodiment of the present application, the object identifier of the participant is determined based on the voice of the participant, and then when the voice subtitles of the participant are displayed, the object identifier of the participant can also be displayed accordingly, and the voice subtitles are displayed in the associated area of ​​the object identifier. This allows the user to intuitively view the participant corresponding to each voice subtitle, facilitates the user to understand the voices of different participants, and improves the efficiency of conference communication.

[0063] In some embodiments of the present application, in order to facilitate the user to intuitively view the actual conference participants corresponding to each voice, after displaying the object identifier and voice subtitles of the conference participants, the above method may further include:

[0064] receiving a fifth input;

[0065] In response to the fifth input, displaying the object identification editing interface;

[0066] receiving a sixth input from a user regarding a first object identifier among the object identifiers of each conference participant;

[0067] In response to the sixth input, the first object identification is updated.

[0068] Among them, the fifth input is an input to the AI ​​interactive interface, and the fifth input is used to display the object identification editing interface, and the fifth input can be the fifth operation. Exemplarily, the fifth input includes but is not limited to: the user's touch input to the AI ​​interactive interface through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the fifth input can be: the user's touch input to the "Character Edit" control in the AI ​​interactive interface.

[0069] The object identification editing interface may be an interface for editing the object identification of each conference member. The object identification of each conference member may be included in the object identification editing interface.

[0070] The first object identifier may be any object identifier among the object identifiers of the conference participants included in the object identifier editing interface.

[0071] The sixth input is an input to the first object identifier, and the sixth input is used to update the first object identifier. The sixth input can be a sixth operation. Exemplarily, the sixth input includes but is not limited to: a touch input of the first object identifier by a user through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the sixth input can be: a click input of the first object identifier by the user.

[0072] In some embodiments of the present application, after the object identifiers of the participating members are displayed, if the user clearly knows who a certain participating member is, the object identifier of the participating member can be edited. Specifically, in response to the fifth input, an object identifier editing interface can be displayed, and then in response to the user's sixth input of the first object identifier in the object identifier editing interface, the first object identifier can be updated.

[0073] Continuing with the above example, refer to Figure 4 , the user clicks the "Role Edit" control 41 to display the object identification editing interface 42, in which there is the object identification of each participant, that is, the object identification "Object A" of participant A and the object identification "Object B" of participant B. If the user knows that participant A is Xiao Ming, the user can click the "Edit Name" control 421 behind the object identification "Object A" of participant A, and an interface 43 for editing the object identification "Object A" of participant A can be displayed. In the interface 43, the user can enter the name "Xiao Ming" of the object identification "Object A" of participant A, and the object identification "Object A" of the participant corresponding to the voice "When does the meeting start?" can be updated to "Xiao Ming".

[0074] It should be noted that the reference Figure 2 , in the display Figure 4 Before clicking the "Role Edit" control 41 shown in the figure, the user can click the floating ball "Function" 22 in the AI ​​interaction interface 20, and then the function controls of each function in the voice recognition function can be displayed, such as Figure 4 The “Role Edit” control 41, the “Smart Reminder” control 45 and the “Summary” control 46.

[0075] In an embodiment of the present application, an object identifier editing interface can be displayed in response to the fifth input, and then the first object identifier can be updated in response to the user's sixth input of the first object identifier in the object identifier editing interface. In this way, the object identifiers of the participating members can be updated according to user needs, so that the user can intuitively view the participating objects corresponding to each voice.

[0076] In some embodiments of the present application, in order to improve the diversity of speech translation texts, after step 110, the above method may further include:

[0077] Receiving a first input of a speech translation text;

[0078] In response to the first input, displaying at least one translation style option;

[0079] receiving a second input for a first translation style option of the at least one translation style option;

[0080] In response to the second input, the speech translation text is updated according to the translation style corresponding to the first translation style option.

[0081] Among them, the first input is an input of the voice translation text, and the above-mentioned first input is used to display at least one translation style option, and the first input can be a first operation. Exemplarily, the above-mentioned first input includes but is not limited to: the user's touch input of the voice translation text through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the above-mentioned first input can be: the user's click input of the voice translation text that needs to change the translation style in the voice translation text of each participant.

[0082] At least one translation style option may be at least one translation style option for voice translation text. For example, the at least one translation style option may include a "better understanding" translation style option and a "more formal" translation style option.

[0083] The first translation style option may be any one of the at least one translation style option.

[0084] The second input is an input of the first translation style option in at least one translation style option, and the second input is used to update the voice translation text according to the translation style corresponding to the first translation style option, and the second input can be a second operation. Exemplarily, the second input includes but is not limited to: the user's touch input of the first translation style option in at least one translation style option through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the second input can be: the user's click input of the first translation style option in at least one translation style option.

[0085] In some embodiments of the present application, when the speech translation text of the speech does not meet the user's needs, the user can adjust the translation style of the speech. Specifically, in response to a first input to the speech translation text, at least one translation style option can be displayed, and then in response to the user's second input to a first translation style option among the at least one translation style option, the speech translation text can be updated according to the translation style corresponding to the first translation style option.

[0086] It should be noted that the above receiving the first input of the speech translation text does not mean that the first input is performed on the speech translation texts of all the conference members, but the first input is performed on the speech translation texts of the speech of each conference member, which the user is not satisfied with. Figure 2 In the example, the user is not satisfied with the voice translation text “When does the meeting start?” of the voice “When does the meeting start?” of the conference participant A, so the user can perform a first input on the voice translation text “When does the meeting start?”.

[0087] In one example, reference Figure 5 In the meeting, a participant said a voice message "Let's not try to boll the ocean with this project". After translating the voice message, the voice translation text obtained is "Let's not try to find a needle in a haystack on this project". The user cannot understand the meaning of the voice translation text "Let's not try to find a needle in a haystack on this project", so the translation style of the voice translation text "Let's not try to find a needle in a haystack on this project" can be updated. Specifically, the user clicks on the voice translation text "Let's not try to find a needle in a haystack on this project" to display a translation style adjustment interface 51. The translation style adjustment interface 51 includes at least one translation style option, such as a "better understanding" translation style option 511 and a "more formal" translation style option 512. The user clicks on the "better understanding" translation style option 511, and the translation style of the voice translation text "Let's not try to find a needle in a haystack on this project" can be updated to a better understood voice translation text "Let's not make this project too complicated".

[0088] In the embodiments of the present application, the translation style of the speech translation text corresponding to the speech can be adjusted and updated according to user needs, so that a speech translation text with a translation style that better meets user needs can be obtained, thereby improving the diversity of the speech translation text.

[0089] In some embodiments of the present application, see Figure 1 , the methods involved above may include:

[0090] Step 110: when receiving the voice of the conference member, display the voice subtitles according to the voice of the conference member; wherein the voice subtitles include the voice translation text, or the voice text and the voice translation text;

[0091] Step 120: Play at least one of the voice of the conference member and the translated voice of the conference member.

[0092] It can be understood that there is no limit to the order in which the display of voice subtitles and the playback of voice are performed.

[0093] In some embodiments of the present application, simultaneous interpretation is supported in a conference, that is, during the voice reception process of each conference member, the voice of each conference member and / or the translated voice of each conference member can also be broadcast.

[0094] The above-mentioned simultaneous interpretation means that the voices of other people can be translated into the voices of the language required by the user for broadcasting, and the voice of the user can be translated into the voices of other people's language for broadcasting. That is, in a meeting, there will be four kinds of voices broadcasting: the user's voice, the translated voice of the user's voice, the voice of the other party, and the translated voice of the other party's voice.

[0095] In an embodiment of the present application, during the voice reception process of each participating member, the voice of each participating member and / or the translated voice of each participating member can be broadcast, so that the user can hear the voice of each participating member and / or the translated voice of each participating member in time, thereby improving the efficiency of conference communication.

[0096] In some embodiments of the present application, in order to improve the diversity of voice broadcasting, before playing at least one of the voice of the conference member and the translated voice of the conference member, the above method may further include:

[0097] receiving a third input;

[0098] In response to a third input, displaying a voice announcement list;

[0099] receiving a fourth input from a user for a first voice broadcast mode among at least one voice broadcast mode;

[0100] The playing of at least one of the voice of the conference member and the translated voice of the conference member may include:

[0101] In response to the fourth input, at least one of the voice of the conference member and the translated voice of the conference member is played in the first voice broadcasting mode.

[0102] Among them, the third input is an input to the AI ​​interactive interface, and the third input is used for the voice broadcast list, and the third input can be a third operation. Exemplarily, the third input includes but is not limited to: the user's touch input to the AI ​​interactive interface through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the third input can be: the user's click input to the "Settings" control in the AI ​​interactive interface.

[0103] The voice broadcast list may be a list showing voice broadcast modes, and the voice broadcast list may include at least one voice broadcast mode.

[0104] When the voices of the conference participants include a first voice of a first object and a second voice of a second object, the language of the first voice is the first language and the language of the second voice is the second language, the language of the translated voice of the first object may be the second language.

[0105] In the above situation, taking the first object as the user corresponding to the electronic device of the voice processing method provided in the embodiment of the present application, and the second object as other participants in the meeting as an example, at least one voice broadcast mode may include: muting the second voice, muting the translation voice corresponding to the second voice, and muting the first voice. In this way, the voice broadcast mode required by the user can be selected according to the user's needs, which improves the flexibility of voice broadcast.

[0106] Muting the second voice may be muting the voice of the second object.

[0107] Muting the translated voice of the second voice may be muting the translated voice corresponding to the voice of the second object.

[0108] Muting the first voice may be muting the voice of the first object.

[0109] The first voice broadcast mode may be any one of at least one voice broadcast mode.

[0110] The fourth input is an input to the first voice broadcast mode, and the fourth input is used to play at least one of the voice of the conference member and the translated voice of the conference member according to the first voice broadcast mode. The fourth input can be a fourth operation. Exemplarily, the fourth input includes but is not limited to: the user's touch input to the first voice broadcast mode through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the fourth input can be: the user's click input to the first voice broadcast mode.

[0111] In some embodiments of the present application, when playing at least one of the voice of a conference member and the translated voice of the conference member, the broadcast can be performed according to the broadcast mode set by the user. Specifically, it can be performed by responding to a third input, displaying a voice broadcast list including at least one voice broadcast mode, and then responding to the user's fourth input of a first voice broadcast mode in at least one voice broadcast mode, playing at least one of the voice of a conference member and the translated voice of the conference member according to the first voice broadcast mode.

[0112] Continue to refer Figure 2 , the user can click on the "settings" control 21, Figure 6 As shown, a voice broadcast list 62 is displayed, and the voice broadcast list 62 includes four voice broadcast modes: "Mute the other party's original voice" 621, "Mute my original voice" 622, "My translation broadcast is muted in this test" 623 and "Synchronous translation and delayed playback" 624. The user can select one of the voice broadcast modes. For example, if the user selects "Mute the other party's original voice" 621, when the user listens to the voice of each participant in the meeting on his or her electronic device, if the language of the voice of other participants except himself or herself is not the language the user is accustomed to, the voice of other participants will be muted, that is, the user cannot hear the voice of other participants on his or her electronic device, and can only hear the translated voice of other participants.

[0113] It should be noted that the above-mentioned "muting the other party's original voice" means muting the original voice of other conference members when the language of the voice of other conference members is not the language that the user is accustomed to, that is, the above-mentioned "muting the second voice". For example, the user's accustomed language is Chinese. If in the meeting, a conference member speaks in a language other than Chinese, such as English, if the user sets "muting the other party's original voice", the voice broadcast heard by the user will not include the English speech of the conference member, but only the translated voice of the conference member's voice, which is in Chinese.

[0114] The above-mentioned "Mute my original voice" means muting the user's own voice, which is the above-mentioned "Mute first voice". In a meeting, if the user sets "Mute my original voice", the voice broadcast heard by the user will not include his own voice.

[0115] The above-mentioned "My translation broadcast is muted in this test" means that the translation of the user's voice is not broadcasted on the user's side, which is the above-mentioned "Mute the translation of the second voice". Because in a meeting, after the user speaks, he does not need to listen to his own translation, but only wants to listen to the translation of other participants, so the user can set "My translation broadcast is muted in this test", so that the user will not hear the translation of his own voice on his side.

[0116] In an embodiment of the present application, the voice broadcast mode can be set according to user needs, so that at least one of the voice of the participant and the translated voice of the participant can be played in the first voice broadcast mode, thereby improving the diversity of voice broadcast.

[0117] In some embodiments of the present application, for the accuracy of simultaneous interpretation, when the first voice is received before the second voice, that is, when the first object speaks before the second object, at least one voice broadcast mode may also include: synchronous translation and delayed playback. The "synchronous translation and delayed playback" is to enable the content of the broadcast to be presented in the order of speech. For example, participant A speaks English voice, and the English voice needs to be translated into Chinese voice for playback, but the translation takes a certain amount of time. During the translation process, participant B speaks Chinese voice, and the Chinese voice can be directly played at the electronic device on the side of participant B. In fact, participant A speaks first and participant B speaks later. Due to the translation delay, during the voice broadcast process, the Chinese voice of participant B may be in front and the translated voice of participant A may be behind. If "synchronous translation and delayed playback" is set, the Chinese voice of participant B can be played with a delay to ensure that the translated voice of participant A is before the Chinese voice of participant B.

[0118] The playing of at least one of the voice of the conference member and the translated voice of the conference member may include:

[0119] When the first voice broadcasting mode is synchronous translation and delayed playback, at least one of the first voice and the translated voice of the first voice is played first, and then the second voice and at least one of the translated voice of the second voice is broadcast.

[0120] In some embodiments of the present application, when the first voice is received before the second voice, that is, when the first object speaks before the second object, when broadcasting at least one of the voices of the conference members and the translated voices of the conference members, if the first voice broadcasting mode is synchronous translation and delayed playback, the first voice and at least one of the translated voices of the first voice are played first, and then the second voice and at least one of the translated voices of the second voice are broadcast. This ensures that at least one of the first voice and the translated voice of the first voice can be broadcast before the second voice and at least one of the translated voices of the second voice.

[0121] It should be noted that after setting the broadcast mode of simultaneous translation and delayed playback, when displaying the voice translation text of the first voice and the voice text of the second voice, the voice translation text of the first voice can be displayed first, and then the voice text of the second voice can be displayed, that is, to ensure that the content displayed on the voice subtitles can also be presented in the order of speaking.

[0122] In an embodiment of the present application, by setting the broadcasting mode of simultaneous translation and delayed playback, when at least one of the voices of the participants and the translated voice is broadcasted, the broadcasting can be performed in the order of the participants' speeches, thereby ensuring the accuracy of simultaneous interpretation.

[0123] In the prior art, in a meeting, if the meeting duration is long, it is difficult for the user to maintain a high degree of concentration throughout the entire meeting, which may cause the user to miss key information.

[0124] In order to solve the above problem, before step 110, the above method may further include:

[0125] receiving a seventh input;

[0126] In response to the seventh input, determining a reminder keyword and a speech reminder member;

[0127] When the voice text of the conference participant includes a reminder keyword and the conference participant is a member who is reminded to speak, the reminder information is displayed.

[0128] Among them, the seventh input is an input to the AI ​​interactive interface, and the seventh input is used to determine the reminder keywords and the members who are reminded to speak, and the seventh input can be the seventh operation. Exemplarily, the seventh input includes but is not limited to: the user's touch input to the AI ​​interactive interface through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiments of the present invention are not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the seventh input can be: the user's click input to the "Smart Reminder" control in the AI ​​interactive interface.

[0129] The reminder keyword may be a word in the voice of a conference participant used to remind the user to pay attention.

[0130] The speech reminding member may be a speaking member used to remind the user's attention, that is, when the speaking member speaks, the user should listen attentively.

[0131] The prompt information may be information generated when the voice text of the conference member includes the reminder keyword and the conference member is a speech reminder member. The prompt information may be used to remind the user that the voice of the speech reminder member includes the reminder keyword.

[0132] In some embodiments of the present application, in order to prevent users from missing important information in a meeting, the words and speakers that the user should pay attention to can be pre-set before receiving the voices of the participating members, that is, wake-up keywords and speech reminder members can be set. When each participating member speaks, the timbre characteristic information of each participating member can be matched with the timbre characteristic information of the set speech reminder member. When it is determined that the timbre characteristic information of a certain speaking member matches the timbre characteristic information of the set speech reminder member, the speech content of the speaker can be matched with the reminder keyword. When it is determined that the speech content of the speaker matches the reminder keyword, for example, the reminder keyword is mentioned in the speech content of the speaker, a prompt message can be generated to remind the user.

[0133] In one example, reference Figure 4 , the user can click on the "Smart Reminder" control 45, and then Figure 7As shown, a smart reminder interface 71 is displayed, in which the user can set the speech reminder members and reminder keywords. For example, the user sets the reminder keywords "report, to-do" and the speech reminder members "Xiao Ming, Xiao Hong". In the subsequent speeches, if "Xiao Ming" and / or "Xiao Hong" mention "report" or "to-do" in their speeches, a prompt message can be generated to remind the user. For example, at the meeting, Xiao Ming's speech is "the next to-do item is to actively promote Project X", then the prompt message "Xiao Ming mentioned the next to-do item" will be generated.

[0134] It should be noted that when matching the speech content of the speech reminder member with the reminder keyword, fuzzy matching can be used, that is, the reminder keyword may not necessarily be mentioned in the speech content of the speech reminder member, but may also be mentioned in the speech content of the speech reminder member. Words similar to the reminder keyword, or the semantics of the speaker's speech content are the same as the reminder keyword. For example, the user sets the reminder keyword "to do", if the speech content of the speech reminder member is "Next, we will actively promote Project X", then the meaning of the speech content is what to do next, which is the same as the reminder keyword "to do", so a prompt message will also be generated to prompt the user.

[0135] In an embodiment of the present application, by setting a speech reminder member and a reminder keyword, when the voice text of a conference member includes a reminder keyword and the conference member is a speech reminder member, a prompt message can be displayed to prevent the user from missing important information.

[0136] In some embodiments of the present application, in order to facilitate the user to trace the speech of each participant in the meeting, the above method may also include:

[0137] In case the meeting ends, receiving an eighth input;

[0138] In response to the eighth input, displaying a meeting minutes generation interface;

[0139] receiving a ninth input for a first meeting minutes type option of the at least one meeting minutes type option;

[0140] In response to the ninth input, a meeting minutes is generated according to the voice of each conference participant according to the meeting minutes type corresponding to the first meeting minutes type option.

[0141] Among them, the eighth input is an input to the AI ​​interactive interface, and the eighth input is used to display the meeting minutes generation interface, and the eighth input can be the eighth operation. Exemplarily, the eighth input includes but is not limited to: the user's touch input to the AI ​​interactive interface through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the eighth input can be: a user's click input to the AI ​​interactive interface.

[0142] The meeting minutes generation interface may be an interface for generating meeting minutes. The meeting minutes generation interface may include at least one meeting minutes type option. The meeting minutes type corresponding to the meeting minutes type option may be the type of the generated meeting minutes.

[0143] The first meeting minute type option may be any one of the at least one meeting minute type option.

[0144] The ninth input is an input for the first meeting minutes generation type option. The ninth input is used to generate meeting minutes according to the meeting minutes type corresponding to the first meeting minutes type option and the voice of each participating member. The ninth input can be the ninth operation. Exemplarily, the ninth input includes but is not limited to: the user's touch input of the first meeting minutes generation type option through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the ninth input can be: the user's click input of the first meeting minutes generation type option.

[0145] In some embodiments of the present application, when the meeting is over, in response to the user's eighth input, a meeting minutes generation interface including at least one meeting minutes type option can be displayed, and then in response to the user's ninth input of a first meeting minutes type option among the at least one meeting minutes type option, a meeting minutes can be generated according to the voice of each participating member according to the meeting minutes type corresponding to the first meeting minutes type option.

[0146] Continue to refer Figure 4 , the user clicks the "Summary" control 46, then Figure 8 The meeting minutes generation interface 81 is shown, which includes two types of meeting minutes generation options: "Full Text Summary" option 811 and "Custom Summary" option 812. The user selects either "Full Text Summary" option 811 or "Custom Summary" option 812. For example, if the "Full Text Summary" option 811 is selected, the meeting minutes can be generated according to the voice of each participating member in the manner corresponding to the full text summary option.

[0147] It should be noted that when generating the meeting minutes, a pre-trained large model may be used to generate the meeting minutes based on the speech translation text corresponding to the speech of each participant in the meeting.

[0148] It should be noted that the generated meeting minutes may be structured meeting minutes, for example, the meeting minutes may include the main contents of the meeting such as the theme, summary, key points, and to-do items.

[0149] In an embodiment of the present application, when the meeting is over, in response to the user's eighth input, a meeting minutes generation interface including at least one meeting minutes type option is displayed, and then in response to the user's ninth input of a first meeting minutes type option among the at least one meeting minutes type option, meeting minutes can be generated according to the meeting minutes type corresponding to the first meeting minutes type option and according to the voice of each participating member, so that subsequent users can trace the speech of each participating member in the meeting, and when generating the meeting minutes, meeting minutes can be generated according to user needs and the meeting minutes type required by the user, thereby improving the flexibility of generating meeting minutes.

[0150] In some embodiments of the present application, in order to further improve the flexibility of generating meeting minutes, generating meeting minutes according to the meeting minutes type corresponding to the first meeting minutes type option and according to the voice of each participating member may include:

[0151] When the meeting minutes type corresponding to the first meeting minutes type option is the first meeting minutes type, the meeting minutes are generated according to the voices of each conference participant according to the first meeting minutes type;

[0152] When the meeting minutes type corresponding to the first meeting minutes type option is the second meeting minutes type, the meeting minutes are generated according to the selected target meeting participants and the voices of the target meeting participants.

[0153] The first meeting minutes type option and the second meeting minutes type option can be two different meeting minutes type options in at least one meeting minutes type option. For example, the first meeting minutes type option here can be the above Figure 8 In the "Full text summary" option 811, the second meeting minutes type option can be Figure 8 "Custom Summary" option in 812.

[0154] The first meeting minutes type may be a meeting minutes type corresponding to the first meeting minutes type option. The first meeting minutes type option may be the above Figure 8 In the case of the "full text summary" option 811, the first meeting minutes type is to organize the voices of all the participants in the meeting to form a meeting minutes containing the speech content of all the participants.

[0155] The second meeting minutes type may be a meeting minutes type corresponding to the second meeting minutes type option. The second meeting minutes type option may be the above Figure 8 In the case of the "custom summary" option 812, the second meeting minutes type is to organize the voices of the selected participating members in the meeting to form a meeting minutes that only contains the speech content of the selected participating members.

[0156] The target conference participants may be conference participants selected by the user for forming the conference minutes.

[0157] In some embodiments of the present application, when the meeting minutes type corresponding to the first meeting minutes type option is the first meeting minutes type, the meeting minutes can be generated according to the voice of each participating member according to the first meeting minutes type.

[0158] Continue to refer Figure 8 , with the first meeting minutes type option as Figure 8 For example, after the user selects the "full text summary" option 811, Fig. 9 As shown, the voices of all the participants in the meeting can be sorted out to form a meeting minutes containing the speech contents of all the participants.

[0159] It should be noted that if Fig. 9 The meeting minutes shown may include the main contents of the meeting, such as the theme, summary, key points, and to-do items.

[0160] When the meeting minutes type corresponding to the first meeting minutes type option is the second meeting minutes type, the meeting minutes are generated according to the selected target meeting participants and the voices of the target meeting participants.

[0161] Continue to refer Figure 8 , with the first meeting minutes type option as Figure 8 For example, after the user selects the "custom summary" option 812, the user can select the target participant. For example, if the selected target participant is Xiao Ming, the following can be done: Fig.10 As shown, Xiao Ming's speech in the meeting is summarized to form a meeting minutes containing only Xiao Ming's speech.

[0162] It should be noted that if Fig.10 The meeting minutes shown may include the main contents of Xiao Ming's speech, such as the subject, summary, key points, and to-do items.

[0163] In the embodiments of the present application, different meeting minutes type options can be selected according to user needs, and meeting minutes of different meeting minutes types can be obtained, thereby improving the flexibility of generating meeting minutes.

[0164] In some embodiments of the present application, the selection of target conference participants may be implemented in the following manner: that is, before generating the meeting minutes according to the selected target conference participants and the voice of the target conference participants, the above method may further include:

[0165] receiving a tenth input for a second object identifier;

[0166] In response to the tenth input, displaying a conference member selection interface;

[0167] In response to the user's eleventh input of the third object identifier in the object identifiers of each conference participant, a target conference participant object is determined.

[0168] The second object identifier may be any object identifier of the multiple speaking objects displayed in the AI ​​interaction interface. For example, the second object identifier may be Figure 2 The object identifier of participant A in the conference is "object A".

[0169] The tenth input is an input of the second object identifier, and the tenth input is used to display the participant selection interface, and the tenth input can be the tenth operation. Exemplarily, the tenth input includes but is not limited to: the user's touch input of the second object identifier through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the tenth input can be: the user's click input of the second object identifier.

[0170] The conference member selection interface may be an interface for selecting target conference members, and the conference member selection interface may include object identifiers of each conference member.

[0171] The third object identifier may be any object identifier among the object identifiers of each conference member included in the conference member selection interface.

[0172] The eleventh input is an input of the third object identifier in the object identifier of each participant. The eleventh input is used to determine the target participant, and the eleventh input can be the eleventh operation. Exemplarily, the eleventh input includes but is not limited to: the user's touch input of the third object identifier in the object identifier of each participant through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be determined according to actual use requirements, and the embodiment of the present invention is not limited. The specific gesture in the embodiment of the present application can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture; the click input in the embodiment of the present application can be a single-click input, a double-click input, or any number of click inputs, etc., and can also be a long press input or a short press input. For example, the eleventh input can be: the user's click input of the third object identifier in the object identifier of each participant.

[0173] In some embodiments of the present application, in response to the tenth input of the second object identifier, a participant selection interface including the object identifiers of each participant can be displayed, and then in response to the user's eleventh input of the third object identifier among the object identifiers of each participant, the participant object corresponding to the third object identifier can be determined as the target participant object.

[0174] Continue to refer Figure 8After the user selects the "Custom Summary" option 812, the Fig.11 As shown, a participant selection interface 111 is displayed, in which the object identifier of each participant in the meeting is included, for example, the object identifier "Xiao Ming" 1111 of the participant Xiao Ming, the object identifier "Xiao Ming" 1112 of the participant Xiao Hong, and the object identifier "Zhang San" 1113 of the participant Zhang San. If the user selects the object identifier "Xiao Ming" 1111 of the participant Xiao Ming, the target participant can be determined to be Xiao Ming, and then after the user clicks the "OK" control 112, the following can be generated: Fig.10 Minutes of meeting shown.

[0175] In an embodiment of the present application, in response to the tenth input of the second object identifier, a participant selection interface including the object identifiers of each participant can be displayed, and then in response to the user's eleventh input of the third object identifier in the object identifiers of each participant, the participant object corresponding to the third object identifier can be determined as the target participant object. In this way, according to user needs, the target participant members involved in the meeting minutes of the meeting minutes type corresponding to the second meeting minutes type option that need to be generated can be selected, thereby improving the flexibility of selecting target participant members.

[0176] It should be noted that after generating the meeting minutes, you can also export the meeting minutes. You can export the meeting minutes in PDF version. For example, you can click Fig. 9 and Fig.10 You can also export the meeting minutes in Word format by clicking the "Export PDF" control in the Fig. 9 and Fig.10 In the "Export Word" control, the specific form of the meeting minutes to be exported can be selected according to user needs and is not limited in the embodiments of the present application.

[0177] The speech processing method provided in the embodiment of the present application can be executed by a speech processing device. In the embodiment of the present application, the speech processing device provided in the embodiment of the present application is described by taking the speech processing method executed by the speech processing device as an example.

[0178] Fig.12 FIG. 1 is a schematic diagram showing the structure of a speech processing device according to an exemplary embodiment. Fig.12 As shown, the speech processing device 1300 may include:

[0179] The display module 1310 is used to display voice subtitles according to the voice of the conference member when receiving the voice of the conference member;

[0180] The voice subtitles include voice translation text, or voice text and voice translation text.

[0181] In an embodiment of the present application, when the conference members are conference members who speak different languages ​​and when the voice of the conference member is received, the translated text of the conference member's voice can be intuitively displayed, or the translated text and voice text of the conference member's voice can be displayed, so that the user can intuitively view the translated text of the conference member's voice, or the translated text and voice text of the conference member's voice, and then communicate with the conference members based on the translated text of the conference member's voice, or the translated text and voice text of the conference member's voice, thereby improving the communication efficiency between conference members who speak different languages.

[0182] In some embodiments of the present application, the display module 1310 is specifically used for:

[0183] The voice subtitles of the conference participants are displayed on the artificial intelligence (AI) interactive interface.

[0184] In some embodiments of the present application, the above-mentioned device may further include:

[0185] A first receiving module, configured to receive a first input of the speech translation text;

[0186] The display module 1310 is further configured to display at least one translation style option in response to the first input;

[0187] A second receiving module, configured to receive a second input from a user on a first translation style option among the at least one translation style option;

[0188] The first updating module is configured to update the speech translation text according to the translation style corresponding to the first translation style option in response to the second input.

[0189] In some embodiments of the present application, the above-mentioned device may further include:

[0190] The voice playing module is used to play at least one of the voice of the conference member and the translated voice of the conference member.

[0191] In some embodiments of the present application, the above-mentioned device may further include:

[0192] A third receiving module, used for receiving a third input;

[0193] The display module 1310 is further configured to display a voice broadcast list in response to the third input, wherein the voice broadcast list includes at least one voice broadcast mode;

[0194] A fourth receiving module, used for receiving a fourth input of a user on a first voice broadcast mode in the at least one voice broadcast mode;

[0195] The voice playback module is specifically used for:

[0196] In response to the fourth input, at least one of the voice of the conference member and the translated voice of the conference member is played in the first voice broadcasting mode.

[0197] In some embodiments of the present application, the voices of the conference participants include a first voice of a first object and a second voice of a second object, the language of the first voice is a first language, the language of the second voice is a second language, and the language of the translated voice of the first object is the second language;

[0198] The at least one voice broadcasting mode includes: muting the second voice, muting a translated voice of the second voice, and muting the first voice.

[0199] In some embodiments of the present application, when the first subject speaks before the second subject, the at least one voice broadcasting mode further includes: simultaneous translation and delayed playback;

[0200] The voice playback module is specifically used for:

[0201] When the first voice broadcasting mode is the synchronous translation and delayed playback, at least one of the first voice and the translated voice of the first voice is played first, and then the second voice and at least one of the translated voice of the second voice is broadcast.

[0202] In some embodiments of the present application, the display module 1310 is specifically used for:

[0203] Determine the object identifier of the conference member according to the voice of the conference member;

[0204] The object identifier of the conference member and the voice subtitle are displayed; the voice subtitle is located in the associated area of ​​the object identifier of the conference member.

[0205] In some embodiments of the present application, the above-mentioned device may further include:

[0206] a fifth receiving module, configured to receive a fifth input;

[0207] The display module 1310 is further configured to display an object identifier editing interface in response to the fifth input, wherein the object identifier editing interface includes the object identifiers of each conference participant;

[0208] A sixth receiving module, used for receiving a sixth input of a first object identifier among the object identifiers of each conference participant by a user;

[0209] The second updating module is used to update the first object identifier in response to the sixth input.

[0210] In some embodiments of the present application, the above-mentioned device may further include:

[0211] a seventh receiving module, configured to receive a seventh input;

[0212] A first determining module, configured to determine a reminder keyword and a speech reminder member in response to the seventh input;

[0213] The display module 1310 is further configured to display prompt information when the voice text of the conference participant includes the reminder keyword and the conference participant is the speech reminder member.

[0214] In some embodiments of the present application, the above-mentioned device may further include:

[0215] An eighth receiving module, used for receiving an eighth input when the meeting ends;

[0216] The display module 1310 is further configured to display a meeting minutes generation interface in response to the eighth input, wherein the meeting minutes generation interface includes at least one meeting minutes type option;

[0217] a ninth receiving module, configured to receive a ninth input of a first meeting minutes type option among at least one meeting minutes type option;

[0218] The meeting minutes generating module is used to generate meeting minutes according to the voice of each conference participant in response to the ninth input according to the meeting minutes type corresponding to the first meeting minutes type option.

[0219] In some embodiments of the present application, the meeting minutes generation module is specifically used to:

[0220] In a case where the meeting minutes type corresponding to the first meeting minutes type option is the first meeting minutes type, generating meeting minutes according to the first meeting minutes type and based on the voices of each conference participant;

[0221] In a case where the meeting minutes type corresponding to the first meeting minutes type option is the second meeting minutes type, meeting minutes are generated according to the selected target meeting participants and the voices of the target meeting participants.

[0222] The speech processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0223] The speech processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0224] The speech processing device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0225] Alternatively, if Fig.13 As shown, an embodiment of the present application also provides an electronic device 1400, including a processor 1401 and a memory 1402, wherein the memory 1402 stores a program or instruction that can be executed on the processor 1401, and when the program or instruction is executed by the processor 1401, the various steps of the above-mentioned speech processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0226] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0227] Fig.14 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0228] The electronic device 1500 includes but is not limited to: a radio frequency unit 1501, a network module 1502, an audio output unit 1503, an input unit 1504, a sensor 1505, a display unit 1506, a user input unit 1507, an interface unit 1508, a memory 1509, and a processor 1510 and other components.

[0229] Those skilled in the art will appreciate that the electronic device 1500 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1510 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.14 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0230] The display unit 1506 is used to display voice subtitles according to the voice of the conference member when receiving the voice of the conference member; wherein the voice subtitles include voice translation text, or voice text and voice translation text.

[0231] In this way, when the conference members are speaking different languages ​​and the voice of the conference members is received, the translated text of the conference member's voice can be intuitively displayed, or the translated text and voice text of the conference member's voice can be displayed, so that the user can intuitively view the translated text of the conference member's voice, or the translated text and voice text of the conference member's voice, and then communicate with the conference members based on the translated text of the conference member's voice, or the translated text and voice text of the conference member's voice, thereby improving the communication efficiency between conference members speaking different languages.

[0232] Optionally, the display unit 1506 is also used to display the voice subtitles of the conference participants on the artificial intelligence (AI) interaction interface.

[0233] In this way, by displaying the voice subtitles of the participants in the AI ​​interactive interface superimposed on the conference interface, the user does not need to exit the conference interface and switch to the AI ​​interactive interface to view the voice subtitles of the participants. That is, the user can intuitively view the voice subtitles of the participants without switching interfaces back and forth, thereby improving the viewing efficiency of the voice subtitles of the participants.

[0234] Optionally, the user input unit 1507 is used to receive a first input of the speech translation text;

[0235] The display unit 1506 is further configured to display at least one translation style option in response to the first input;

[0236] The user input unit 1507 is further used to receive a second input of a user on a first translation style option among the at least one translation style option;

[0237] The processor 1510 is configured to update the speech translation text according to the translation style corresponding to the first translation style option in response to the second input.

[0238] In this way, the translation style of the speech translation text corresponding to the speech can be adjusted and updated according to user needs, so that a speech translation text with a translation style that better meets user needs can be obtained, thereby improving the diversity of the speech translation text.

[0239] Optionally, the audio output unit 1503 is used to play at least one of the voice of the conference member and the translated voice of the conference member.

[0240] In this way, during the voice reception process of each participant, the voice of each participant and / or the translated voice of each participant can be broadcast, so that users can hear the voice of each participant and / or the translated voice of each participant in time, thereby improving the efficiency of conference communication.

[0241] Optionally, the user input unit 1507 is further configured to receive a third input;

[0242] The display unit 1506 is further configured to display a voice broadcast list in response to the third input, wherein the voice broadcast list includes at least one voice broadcast mode;

[0243] The user input unit 1507 is further used to receive a fourth input from the user for the first voice broadcast mode in the at least one voice broadcast mode;

[0244] The audio output unit 1503 is further configured to respond to the fourth input and play at least one of the voice of the conference member and the translated voice of the conference member in the first voice broadcasting manner.

[0245] In this way, the voice broadcast mode can be set according to user needs, so that at least one of the voice of the conference member and the translated voice of the conference member can be played in the first voice broadcast mode, thereby improving the diversity of voice broadcast.

[0246] Optionally, the voices of the conference members include the first voice of the first object and the second voice of the second object, the language of the first voice is the first language, the language of the second voice is the second language, and the language of the translated voice of the first object is the second language; the at least one voice broadcasting method includes: muting the second voice, muting the translated voice of the second voice, and muting the first voice.

[0247] In this way, the voice broadcast method required by the user can be selected according to the user's needs, which improves the flexibility of the voice broadcast.

[0248] Optionally, when the first object speaks before the second object, the at least one voice broadcasting mode also includes: simultaneous translation and delayed playback; the audio output unit 1503 is also used to first play at least one of the first voice and the translated voice of the first voice, and then broadcast the second voice and at least one of the translated voice of the second voice when the first voice broadcasting mode is the simultaneous translation and delayed playback.

[0249] In this way, by setting the broadcasting mode of simultaneous translation and delayed playback, when at least one of the voice of the participants and the translated voice is broadcast, it can be broadcast in the order of the participants' speeches to ensure the accuracy of simultaneous interpretation.

[0250] Optionally, the processor 1510 is further configured to determine the object identifier of the conference member according to the voice of the conference member;

[0251] The display unit 1506 is further used to display the object identifier of the conference member and the voice subtitle; the voice subtitle is located in the associated area of ​​the object identifier of the conference member.

[0252] In this way, by determining the object identifier of the participant based on the voice of the participant, and then displaying the voice subtitles of the participant, the object identifier of the participant can also be displayed accordingly, and the voice subtitles can be displayed in the associated area of ​​the object identifier. In this way, the user can intuitively view the participant corresponding to each voice subtitle, which is convenient for the user to understand the voices of different participants and improves the efficiency of conference communication.

[0253] Optionally, the user input unit 1507 is further configured to receive a fifth input;

[0254] The display unit 1506 is further configured to display an object identifier editing interface in response to the fifth input, wherein the object identifier editing interface includes the object identifiers of each conference participant;

[0255] The user input unit 1507 is further used to receive a sixth input of the user for the first object identifier in the object identifiers of each conference participant;

[0256] The processor 1510 is further configured to update the first object identifier in response to the sixth input.

[0257] In this way, by responding to the fifth input, the object identifier editing interface can be displayed, and then in response to the user's sixth input of the first object identifier in the object identifier editing interface, the first object identifier can be updated. In this way, the object identifiers of the participating members can be updated according to user needs, so that the user can intuitively view the participating objects corresponding to each voice.

[0258] Optionally, the user input unit 1507 is further configured to receive a seventh input;

[0259] The processor 1510 is further configured to determine a reminder keyword and a speech reminder member in response to the seventh input;

[0260] The display unit 1506 is further configured to display prompt information when the voice text of the conference participant includes the reminder keyword and the conference participant is the speech reminding member.

[0261] Thus, by setting the speech reminder member and the reminder keyword, when the voice text of the conference member includes the reminder keyword and the conference member is the speech reminder member, the reminder information can be displayed to prevent the user from missing important information.

[0262] Optionally, the user input unit 1507 is further configured to receive an eighth input when the conference ends;

[0263] The display unit 1506 is further configured to display a meeting minutes generation interface in response to the eighth input, wherein the meeting minutes generation interface includes at least one meeting minutes type option;

[0264] The user input unit 1507 is further used to receive a ninth input of a first meeting minutes type option among the at least one meeting minutes type option;

[0265] Processor 1510 is further configured to generate meeting minutes according to the voice of each conference participant and in response to the ninth input, according to the meeting minutes type corresponding to the first meeting minutes type option.

[0266] In this way, when the meeting is over, in response to the user's eighth input, a meeting minutes generation interface including at least one meeting minutes type option is displayed, and then in response to the user's ninth input to a first meeting minutes type option among the at least one meeting minutes type option, meeting minutes can be generated according to the meeting minutes type corresponding to the first meeting minutes type option and according to the voice of each participant, so that subsequent users can trace the speech of each participant in the meeting, and when generating meeting minutes, meeting minutes can be generated according to user needs and the meeting minutes type required by the user, thereby improving the flexibility of generating meeting minutes.

[0267] Optionally, processor 1510 is further used to generate meeting minutes according to the voice of each participating member according to the first meeting minutes type when the meeting minutes type corresponding to the first meeting minutes type option is the first meeting minutes type; and to generate meeting minutes according to the voice of the target participating member selected according to the selected target participating member when the meeting minutes type corresponding to the first meeting minutes type option is the second meeting minutes type.

[0268] In this way, different meeting minutes type options can be selected according to user needs, and meeting minutes of different meeting minutes types can be obtained, thus improving the flexibility of generating meeting minutes.

[0269] It should be understood that in the embodiment of the present application, the input unit 1504 may include a graphics processor (Graphics Processing Unit, GPU) 15041 and a microphone 15042, and the graphics processor 15041 processes the image data of the static picture or video obtained by the image capture device (such as a color camera) in the video capture mode or the image capture mode. The display unit 1506 may include a display panel 15061, and the display panel 15061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1507 includes a touch panel 15071 and at least one of other input devices 15072. The touch panel 15071 is also called a touch screen. The touch panel 15071 may include two parts: a touch detection device and a touch controller. Other input devices 15072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0270] The memory 1509 can be used to store software programs and various data. The memory 1509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1509 may include a volatile memory or a non-volatile memory, or the memory 1509 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0271] The processor 1510 may include one or more processing units; optionally, the processor 1510 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1510.

[0272] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned speech processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0273] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0274] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned speech processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0275] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0276] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned speech processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0277] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0278] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0279] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A speech processing method, characterized in that: The method comprises: When receiving the voice of a conference member, displaying voice subtitles according to the voice of the conference member; The voice subtitles include voice translation text, or voice text and voice translation text.

2. The method according to claim 1, characterized in that The displaying of voice subtitles according to the voice of the conference participant includes: The voice of the conference participants is displayed in the artificial intelligence (AI) interactive interface with voice subtitles.

3. The method according to claim 1, characterized in that The method further comprises: Receiving a first input of the speech translation text; In response to the first input, displaying at least one translation style option; receiving a second input from a user regarding a first translation style option among the at least one translation style option; In response to the second input, the speech translation text is updated according to the translation style corresponding to the first translation style option.

4. The method according to claim 1, characterized in that: The method further comprises: At least one of the voice of the conference member and the translated voice of the conference member is played.

5. The method according to claim 4, characterized in that The method further comprises: receiving a third input; In response to the third input, displaying a voice broadcast list, wherein the voice broadcast list includes at least one voice broadcast mode; Receiving a fourth input from a user for a first voice broadcast mode among the at least one voice broadcast mode; The playing of at least one of the voice of the conference member and the translated voice of the conference member includes: In response to the fourth input, at least one of the voice of the conference member and the translated voice of the conference member is played in the first voice broadcasting mode.

6. The method according to claim 5, characterized in that The voices of the conference participants include a first voice of a first object and a second voice of a second object, the language of the first voice is a first language, the language of the second voice is a second language, and the language of the translated voice of the first object is the second language; The at least one voice broadcasting mode includes: muting the second voice, muting a translated voice of the second voice, and muting the first voice.

7. The method according to claim 6, characterized in that In the case where the first subject speaks before the second subject, the at least one voice broadcasting mode further includes: simultaneous translation and delayed playback; The playing of at least one of the voice of the conference member and the translated voice of the conference member includes: When the first voice broadcasting mode is the synchronous translation and delayed playback, at least one of the first voice and the translated voice of the first voice is played first, and then the second voice and at least one of the translated voice of the second voice is broadcast.

8. The method according to claim 1, characterized in that The displaying of voice subtitles according to the voice of the conference participant includes: Determine the object identifier of the conference member according to the voice of the conference member; The object identifier of the conference member and the voice subtitle are displayed; the voice subtitle is located in the associated area of ​​the object identifier of the conference member.

9. The method according to claim 8, characterized in that The method further comprises: receiving a fifth input; In response to the fifth input, displaying an object identification editing interface, wherein the object identification editing interface includes the object identification of each conference participant; receiving a sixth input from a user regarding a first object identifier among the object identifiers of each conference participant; In response to the sixth input, the first object identification is updated.

10. The method according to claim 1, characterized in that The method further comprises: receiving a seventh input; In response to the seventh input, determining a reminder keyword and a speech reminder member; When the voice text of the conference participant includes the reminder keyword and the conference participant is the speech reminder member, the reminder information is displayed.

11. The method according to claim 1, characterized in that: The method further comprises: In case the meeting ends, receiving an eighth input; In response to the eighth input, displaying a meeting minutes generation interface, wherein the meeting minutes generation interface includes at least one meeting minutes type option; receiving a ninth input for a first meeting minutes type option of the at least one meeting minutes type option; In response to the ninth input, a meeting minutes is generated according to the meeting minutes type corresponding to the first meeting minutes type option and based on the voices of the various conference participants.

12. The method according to claim 11, characterized in that Generating the meeting minutes according to the meeting minutes type corresponding to the first meeting minutes type option and according to the voice of each conference participant includes: In a case where the meeting minutes type corresponding to the first meeting minutes type option is the first meeting minutes type, generating meeting minutes according to the first meeting minutes type and based on the voices of each conference participant; In a case where the meeting minutes type corresponding to the first meeting minutes type option is the second meeting minutes type, meeting minutes are generated according to the selected target meeting participants and the voices of the target meeting participants.

13. A speech processing device, characterized in that: The device comprises: A display module, configured to display voice subtitles according to the voice of a conference member when receiving the voice of the conference member; The voice subtitles include voice translation text, or voice text and voice translation text.

14. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the speech processing method according to any one of claims 1 to 12 are implemented.

15. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the speech processing method according to any one of claims 1 to 12 are implemented.

Images