Audio input method, electronic device, storage medium and computer program product
By switching the input mode using the second operation during the voice input process, the problem that voice input only supports a single output mode in the prior art is solved, and diverse outputs and efficient operations are achieved.
Patent Information
- Application Number
- CN202410986525.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-22
AI Technical Summary
The existing voice input technology only supports a single output mode, which is difficult to meet the diversity needs of users when inputting content through the same control, resulting in low user operation efficiency.
By determining the input mode using a second operation when the user acquires audio, thereby switching different output modes during the audio acquisition process, supporting a combination of multiple input and output modes, such as converting voice into text, expressions or images.
It improves user's operation convenience and input efficiency, supports diverse outputs, and reduces the complex operations that users need to manually edit after input.
Smart Images

Figure CN118945562A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to an audio input method, an electronic device, a storage medium, and a computer program product. Background Art
[0002] With the development of speech recognition technology and Internet technology, users can more conveniently use voice input instead of keyboard input to improve input efficiency. For example, when using various applications, users can trigger the audio input control and speak what they want to input. After the voice input is completed, the user's speech will be converted into text and displayed in the interface. Summary of the invention
[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0004] According to some embodiments of the present disclosure, an audio input method is provided, comprising: collecting audio in response to a first operation of a user on an audio input control; determining an input mode according to a second operation of the user during the process of the user performing the first operation on the audio input control, wherein different input modes correspond to different output modes; and outputting content corresponding to the audio according to the output mode corresponding to the input mode.
[0005] According to some embodiments of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the audio input method of any embodiment described in the present disclosure based on instructions stored in the memory.
[0006] According to some embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the audio input method of any embodiment described in the present disclosure is performed.
[0007] According to some embodiments of the present disclosure, a computer program product is provided. When the computer program product is run on a computer, the computer implements the audio input method of any embodiment described in the present disclosure.
[0008] Other features, aspects and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The preferred embodiments of the present disclosure are described below with reference to the accompanying drawings. The drawings described herein are used to provide a further understanding of the present disclosure. Each of the drawings together with the following detailed description is included in this specification and forms a part of the specification, and is used to explain the present disclosure. It should be understood that the drawings described below only relate to some embodiments of the present disclosure and do not constitute a limitation on the present disclosure. In the drawings:
[0010] Figure 1 A flowchart of an audio input method according to some embodiments of the present disclosure is shown.
[0011] Figure 2A and 2B A schematic diagram of an operation interface according to some embodiments of the present disclosure is shown.
[0012] Figure 3 A schematic flow chart of an input mode determination method according to some embodiments of the present disclosure is shown.
[0013] Figure 4 A schematic structural diagram of an audio input device according to some embodiments of the present disclosure is shown.
[0014] Figure 5 A schematic structural diagram of an electronic device according to some embodiments of the present disclosure is shown.
[0015] Figure 6 A schematic diagram of the structure of a computer system according to some embodiments of the present disclosure is shown.
[0016] It should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not necessarily drawn according to the actual proportional relationship. The same or similar reference numerals are used in the various drawings to represent the same or similar parts. Therefore, once an item is defined in one drawing, it may not be further discussed in subsequent drawings. DETAILED DESCRIPTION
[0017] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure, but it is obvious that the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The following description of the embodiments is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. It should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein.
[0018] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be performed in different orders, and / or performed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown in the execution. The scope of the present disclosure is not limited in this respect. Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values of the parts and steps set forth in these embodiments should be interpreted as being merely exemplary and do not limit the scope of the present disclosure.
[0019] The term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". In addition, the term "include" and its variations used in the present disclosure mean an open term that includes at least the following elements / features but does not exclude other elements / features, that is, "including but not limited to". Therefore, include is synonymous with include. The term "based on" means "based at least in part on".
[0020] References throughout this specification to "one embodiment," "some embodiments," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. For example, the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; and the term "some embodiments" means "at least some embodiments." Furthermore, the appearances of the phrases "in one embodiment," "in some embodiments," or "in an embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment, but may do so.
[0021] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. Unless otherwise specified, the concepts of "first", "second", etc. are not intended to imply that the objects described in this way must be in a given order in time, space, ranking, or any other manner.
[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0024] The embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings, but the present disclosure is not limited to these specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures or characteristics can be combined in any suitable manner that will be clear from the present disclosure by a person of ordinary skill in the art.
[0025] At present, voice input is often used in chat scenarios. When the text corresponding to the user's speech content is presented to the other party of the chat, it is also reflected in the user's speech content. For example, when user A and user B are chatting, regardless of whether user A says "Have you eaten" or "Smile" through the audio input control, the text that user A has sent the message "Have you eaten" or "Smile" will be directly displayed in the chat interface. Although smile is the expression that user A wants to send to user B, or user A's current mood, the text of "Smile" will still appear in the form of user dialogue, and it looks no different from "Have you eaten". That is, in the related art, the content input by the user through voice is output through a single mode.
[0026] As users' expressions become more and more diverse, a single output mode can no longer meet their needs. Therefore, whether it is a social application, search application or other types of applications, users can output in a variety of ways. For example, input text, input voice, input emoticons, input images, input videos, etc., or users can trigger input through different controls. These inputs obtained through different input methods will be displayed on the interface in the corresponding way.
[0027] However, for the content input by the user through the same control, only one output mode is still supported. For example, in an application with a dialogue function, in addition to the user directly sending voice, when the user inputs through the audio input control, the voice can only be recognized as text and the text is directly displayed in the interface. If the user's speech content is only an expression of the user's current psychological activity, or an action described by the user in language, the user needs to call the keyboard again and manually add symbols to the corresponding content. For example, if the user says "Hello, smile", "Hello, smile" will be directly displayed on the interface. The user needs to add brackets to the smile to indicate that "smile" is his current psychological activity. For example, the content manually edited by the user is "Hello (smile)". Alternatively, the user says "Hello" to convert it into text, and then manually adds a smiling expression, or sends an image representing a smile. Therefore, for the content input by the user's voice, the user needs to perform further complex operations to adjust the corresponding text to different display modes. In this way, the user's operation efficiency is relatively low.
[0028] Therefore, a technical problem to be solved by the present disclosure is: how to improve operation efficiency while increasing the diversity of user input.
[0029] In order to solve the above problems, the present disclosure provides an audio input method. In the process of collecting audio such as the user's voice, the user can control the input mode through additional operations so that the audio is output in the mode expected by the user. Figure 1 An embodiment of the audio input method of the present disclosure is described.
[0030] Figure 1 FIG. 2 shows a flow chart of an audio input method according to some embodiments of the present disclosure. Figure 1 As shown, the audio input method of this embodiment includes steps S102 to S106.
[0031] In step S102, in response to a first operation of the user on the audio input control, audio is collected.
[0032] The audio input control is a control used to trigger the collection of audio, such as the user's voice, the song sung by the user, and so on. It can be a control on the interface of the application, or a physical control on the user device that hosts the application. The user can trigger the control or physical control on the interface through a first operation such as clicking or long pressing. The user can also trigger the audio input control through a specified gesture or action, such as shaking the user device or performing a specified gesture operation on the screen to trigger various audio input controls to start audio collection.
[0033] In some embodiments, the first operation is a long press operation. The long press operation can avoid erroneous collection caused by the user's accidental touch or the friction caused by the user device being in the user's bag. While the user maintains the long press operation, the audio that the user needs to record can be continuously collected, and when the user ends the long press, it can be regarded as ending this audio collection.
[0034] The embodiments of the present disclosure support the situation where the user performing the first operation and the speaking user are the same user, that is, in response to the user's first operation on the audio input control, the audio of the user is collected. In addition, it also supports the situation where the user performing the first operation and the speaking user are different users, that is, in response to the first user's first operation on the audio input control, the audio of the second user is collected. In actual applications, there is a situation where user A performs the first operation and collects the audio of user B with the consent of user B, which can facilitate user A to assist user B who is currently inconvenient to operate to collect audio.
[0035] In step S104, during the process of the user performing a first operation on the audio input control, an input mode is determined according to a second operation of the user, wherein different input modes correspond to different output modes.
[0036] That is, during the process of collecting audio, the user can determine the input mode through the second operation. For example, the current input mode can be switched to another input mode in response to the second operation. Such as switching the first input mode to the second input mode, or switching the second input mode to the first input mode, etc. In this way, during an audio collection process, the user can switch the input mode by executing the second operation, without the need for the user to manually edit or modify after the collection is completed. This makes it convenient for the user to determine the output mode of the speech content during the speaking process, thereby improving the user's operating efficiency.
[0037] The second operation can be implemented in a variety of ways, such as operation on controls in the interface, gesture operation (such as sliding gesture, etc.), operation on physical controls of the user device, operation on the entire user device (such as shaking, moving, rotating, etc.), etc. For sliding gesture operation, it can be in any direction or in a specified direction.
[0038] In some embodiments, the first operation is a long press operation, and the second operation is a sliding gesture operation. For example, the user triggers the audio acquisition by long pressing the audio input control, and then during the long press, the sliding gesture operation can be performed at the same time. That is, the finger that is long pressed by sliding can maintain the long press and perform the sliding gesture at the same time, which is highly operable. In this way, the user can determine the input mode more efficiently while speaking.
[0039] Of course, the first operation and the second operation may also be other combinations. For example, the user may deflect the angle of the user device while long pressing the audio input control, or may click the screen while shaking the user device to trigger audio collection, etc. The present disclosure does not limit this.
[0040] The input mode is used to distinguish the audio input by the user, so that the audio can be subsequently converted into content corresponding to the input mode and output in the output mode corresponding to the input mode. There are multiple input modes, and the number can be set as needed. Multiple input modes correspond to multiple output modes one by one.
[0041] In some embodiments, the input mode includes a first input mode and a second input mode, the first input mode is used to input dialogue content, and the second input mode is used to input non-dialogue content. Non-dialogue content, for example, includes at least one of text, expression, and image. When the non-dialogue content is text, it is used to describe at least one of the user's psychology, emotion, action, behavior, etc. As needed, the input mode can also be more in number or of other types. For example, multiple input modes can be respectively directed to multiple specified objects, the first input mode is the speech content to object A, the second input mode is the speech content to object B, the third input mode is the speech content to object C, and so on.
[0042] Determining the input mode according to the second operation of the user may be determining the input mode according to whether the second operation is performed. In addition, the second operation may further include multiple types or have multiple parameters and other attributes, so that the input mode is determined according to the attributes of the second operation. For example, when the second operation is a sliding gesture operation, sliding in different directions may correspond to the same input mode or may correspond to different input modes.
[0043] During one audio collection process, the user can perform the second operation once or multiple times, thereby enabling one or multiple input mode switching during one audio collection process, thereby increasing the diversity of the user's output and simplifying the operation.
[0044] In step S106, the content corresponding to the audio is output according to the output mode corresponding to the input mode.
[0045] The output mode is used to describe the type, display style, etc. of the content corresponding to the collected audio, so that the audio corresponding to different input modes can be distinguished when output. The content corresponding to the audio may include at least one of text, expression, image, sound, and control information. For example, the audio can be converted into text first, and then the text can be converted into content corresponding to the output mode according to the output mode. The control information can be control information for the functions of the application, or it can be control information for the objects in the application, such as controlling the virtual objects in the application to perform operations corresponding to the content in the audio (such as actions, expressions, appearance, etc.).
[0046] The first output mode is used to directly display the text corresponding to the audio, and the second output mode is used to display the text corresponding to the audio identified by a designated symbol, or to display non-text information associated with the text corresponding to the audio.
[0047] For example, in the audio content, the content corresponding to the second input mode is identified by a specified symbol, and the content corresponding to the first input mode is not identified by a symbol. Suppose the text corresponding to the collected audio is "Hello, smile". Based on the user's second operation, it is determined that "Hello" is the first output mode and "Smile" is the second output mode. The output content can be the text "Hello (Smile)". Of course, in some embodiments, the output content can also be the text "Hello" and an expression representing a smile, or the text "Hello" can be displayed and the virtual object in the current interface can be controlled to execute a smiling expression or display an image of a smile of the virtual object. Other output modes can also be used as needed, which will not be repeated here.
[0048] During the audio collection, the user's spoken content can be presented on the interface in real time in a corresponding output mode. That is, before the audio collection is completed, the content corresponding to the user's audio can be output. Of course, the corresponding content can also be displayed on the interface after the audio collection is completed.
[0049] Through the above embodiments, in the process of collecting the user's audio by using the first operation, the user can control the input mode by the second operation so that the audio is output in the output mode corresponding to the input mode. Thus, while supporting the output diversity of the user, the user's operation convenience is improved. Therefore, the embodiments of the present disclosure can improve the user's input efficiency.
[0050] The operation position of some types of first operations overlaps with the audio input control. In some embodiments, the initial input mode is determined according to the trigger area of the first operation performed by the user in the audio input control. That is, the first input mode after the user triggers the audio input control. For example, in response to the trigger position of the first operation being the first area of the audio input control, the initial input mode is determined to be the first input mode; in response to the trigger position of the first operation being the second area of the audio input control, the initial input mode is determined to be the second input mode, and so on.
[0051] Of course, a default initial input mode can also be set. That is, no matter where the user triggers the audio input control, a certain input mode can be defaulted as the initial input mode. Those skilled in the art can make a selection as needed.
[0052] The following is an exemplary description of how to determine the input mode according to the second operation of the user.
[0053] In some embodiments, during the process in which the user performs a first operation on the audio input control, in response to a second operation performed by the user being a designated operation, the current input mode is switched to another input mode.
[0054] That is, the second operation may include one or more operations, such as multiple sub-operations, multiple types of operations, or operations with multiple attributes. It is possible to pre-set which specified operations can trigger the switching of the current input mode. For example, the switching may be triggered in response to the user performing a sliding gesture in any direction, or the switching may be triggered only when the user performs a sliding gesture in a specified direction, or the switching may be triggered only when the user performs a sliding gesture in a specified direction and the current input mode is a specified input mode.
[0055] For example, in response to the current input mode being the first input mode and the second operation performed by the user being the first designated operation, the current input mode is switched to the second input mode. Assuming that the current mode is used to input dialogue content, in response to the second operation performed by the user being the first designated operation, the input mode is switched to a mode for inputting non-dialogue content.
[0056] In addition, the current second input mode may be switched back to the first input mode. In some embodiments, in response to the current input mode being the second input mode and the second operation performed by the user being the second designated operation, the current input mode is switched to the first input mode.
[0057] In one example, the first designated operation and the second designated operation are sliding gesture operations with different sliding directions. Figure 2A and 2B An embodiment of switching between different input modes through a slide gesture is described.
[0058] Figure 2A and 2B Schematic diagram of an operation interface according to some embodiments of the present disclosure is shown. Figure 2A As shown, an audio input control 21 is provided in the interface 2. The user can trigger audio collection by long pressing the control 21, and end the audio collection when the long pressing operation ends.
[0059] The user long presses the control 21 and starts speaking. Figure 2B As shown. Assume that the initial input mode is the first input mode. Then, the user performs a sliding gesture (e.g., swiping right) to switch the input mode to the second input mode. After that, the user performs a sliding gesture in a different direction (e.g., swiping left) to switch the input mode back to the first input mode. The user can also perform any number of gesture operations during the speaking process. Finally, the user ends the long press, and the audio collection process ends.
[0060] In some embodiments, the user may also swipe right to switch the input mode from the first input mode to the second input mode, and then swipe left to switch back to the first input mode. That is, swiping in any direction may switch from the first mode to the second input mode, and then performing the opposite swiping operation may switch from the second input mode back to the first input mode.
[0061] In the case of displaying an audio input control, the audio input control may have a variety of display styles. For example, in different input modes, the style of the audio input control may be different. Taking FIG. 2 as an example, in different input modes, the control 21 may display different colors to prompt the user of the current input mode. As needed, text prompts may also be displayed in the interface to make it clear to the user which input mode is currently in, or what operation can be used to switch the input mode, etc.
[0062] In some embodiments, the correspondence between the designated area of the control 21 and the initial input mode can also be set. For example, when the user triggers the control 21 through the first operation, when the user touches the middle part of the control 21, the first input mode can be used as the initial input mode. When the user initially touches both sides of the control 21, the initial input mode can be determined as the second input mode. Of course, it is also possible to set that any area of the control 21 touched by the user triggers the default input mode, such as the first input mode, and then execute the switching of the input mode in response to the second operation of the user. For example, switch to the second input mode.
[0063] After the audio acquisition is finished, an example of the output result is as follows Figure 2B As shown in the display control 22. In this example, the dialogue content that the user wants to express is displayed in the interface without symbols, while the action or psychological activity that the user wants to express is placed in brackets for display.
[0064] The following takes the content in control 22 as an example to exemplarily describe the user's speaking and operation process. After long pressing control 21, the user can say the following content and perform the following actions, that is, form the following speaking-action sequence, and the user's actions are expressed in "<>": <Swipe left>-Smile-<Swipe right>-Are you back? I'm cooking, I'll talk to you later-<Swipe left>-Turn around and cook. Through the first left swipe operation, the input mode is switched to the input mode of non-dialogue content, and then the "smile" spoken by the user is determined as non-dialogue content; through the first right swipe operation, the input mode is switched to the input mode of dialogue content, and then the user's "Are you back? I'm cooking, I'll talk to you later" is determined as dialogue content; through the second left swipe operation, the input mode is switched again to the input mode of non-dialogue content, and then the user's "Turn around and cook" is determined as non-dialogue content. Then, the user ends the long press operation. The above-mentioned voice determines the corresponding output content according to the input mode of each part, and in Figure 2B In the example output result, "(smile) Are you back? I'm cooking, I'll talk to you later (then turns around to cook)", that is, the dialogue content is displayed directly, and the non-dialogue content is displayed in brackets.
[0065] Some embodiments of the present disclosure support converting the user audio during an audio collection process into content corresponding to multiple input modes. That is, the user's audio is divided into multiple sections, and these sections involve multiple output modes. In this case, it can be determined which part of the audio is output in which output mode according to the time when the user performs one or more second operations. Figure 3 An embodiment of the input mode determination method of the present disclosure is described.
[0066] Figure 3 FIG. 2 shows a flow chart of a method for determining an input mode according to some embodiments of the present disclosure. Figure 3 As shown, the input mode determination method of this embodiment includes steps S302 to S304.
[0067] In step S302, when the user performs a first operation on the audio input control, the audio or content corresponding to the audio is divided into a plurality of sections according to the execution time of the user's second operation.
[0068] The execution time can be directly used as the segmentation point for segmenting the audio, or the segmentation point for segmenting the audio can be determined with reference to the execution time of the second operation, but does not have to be completely consistent with it.
[0069] Considering that the user's action and the time point when the user wants to switch when speaking may not completely coincide in time, in some embodiments, the segmentation point of the paragraph can be more accurately determined with the help of the time when the user performs the second operation and the semantics corresponding to the audio.
[0070] For example, the audio or the content corresponding to the audio can be divided into multiple units according to the semantics of the audio. For example, after the audio is converted into text, the text can be input into a text processing model to divide the text into multiple units. Each unit can be a sentence, a clause, a phrase, a word, etc. Alternatively, the user's audio can be directly input into the audio processing model and the segmented result output by the audio processing model can be directly obtained. Then, according to the execution time of the user's second operation, the multiple units are divided into multiple paragraphs.
[0071] For example, when a user says "Smile, long time no see, how are you doing recently?", the user's action is slightly delayed after saying "smile", and the second action is performed after saying "OK". If the segmentation is performed completely according to the time point, "smile, OK" will be determined as one output mode, and "long time no see, how are you doing recently" will be determined as another output mode. However, by using semantic segmentation, "smile" is regarded as one unit and "long time no see" is regarded as another unit, and the segmentation point is determined between "smile" and "long time no see". As a result, the user's input intention can be identified more accurately.
[0072] In step S304, the input mode of each paragraph is determined according to the input mode corresponding to the second operation of the user.
[0073] For example, the second operation closest to the starting point of each paragraph may be associated with the paragraph, and the input mode after the second operation is performed may be determined as the input mode of the paragraph.
[0074] Then, the content corresponding to the audio may be output according to the output mode corresponding to the input mode of each paragraph.
[0075] The above embodiment divides the audio or the content corresponding to the audio into paragraphs according to the time when the user performs the second operation, and then determines the input mode of each paragraph. Therefore, it is possible to support the use of multiple input modes and output modes in a single audio collection process, which facilitates the user to freely switch the input mode during speaking, thereby improving the richness of audio input and the convenience of operation.
[0076] The above is an introduction to the various method embodiments of the present disclosure. The following further describes the devices used to perform the above methods.
[0077] Figure 4 FIG. 2 shows a schematic diagram of the structure of an audio input device according to some embodiments of the present disclosure. Figure 4As shown, the audio input device 4 of this embodiment includes: a collection module 401, configured to collect audio in response to a first operation of a user on an audio input control; a determination module 402, configured to determine an input mode according to a second operation of the user during the process of the user performing a first operation on the audio input control, wherein different input modes correspond to different output modes; and an output module 403, configured to output content corresponding to the audio according to an output mode corresponding to the input mode.
[0078] In some embodiments, the first operation is a long press operation.
[0079] In some embodiments, the second operation includes a slide gesture operation.
[0080] In some embodiments, the second operation includes a sliding gesture operation along a specified direction.
[0081] In some embodiments, the determination module 402 is further configured to determine an initial input mode according to a triggering area of a first operation performed by a user in the audio input control.
[0082] In some embodiments, the determination module 402 is further configured to: during the process of the user performing a first operation on the audio input control, in response to a second operation performed by the user being a designated operation, switch the current input mode to another input mode.
[0083] In some embodiments, the determination module 402 is further configured to: in response to the current input mode being the first input mode and the second operation performed by the user being the first designated operation, switch the current input mode to the second input mode.
[0084] In some embodiments, the determination module 402 is further configured to: in response to the current input mode being the second input mode and the second operation performed by the user being the second designated operation, switch the current input mode to the first input mode.
[0085] In some embodiments, the first designated operation and the second designated operation are sliding gesture operations with different sliding directions.
[0086] In some embodiments, the determination module 402 is further configured to: during the process of the user performing a first operation on the audio input control, divide the audio or the content corresponding to the audio into multiple paragraphs according to the execution time of the user's second operation; and determine the input mode of each paragraph according to the input mode corresponding to the user's second operation.
[0087] In some embodiments, the determination module 402 is further configured to: divide the audio or content corresponding to the audio into multiple units according to the semantics of the audio; and divide the multiple units into multiple paragraphs according to the execution time of the user's second operation.
[0088] In some embodiments, the output module 403 is further configured to output content corresponding to the audio according to an output mode corresponding to the input mode of each paragraph.
[0089] In some embodiments, the output mode includes a first output mode and a second output mode; the first output mode is used to directly display the text corresponding to the audio; the second output mode is used to display the text corresponding to the audio identified by a specified symbol, or to display non-text information associated with the text corresponding to the audio.
[0090] In some embodiments, the input mode includes a first input mode and a second input mode, the first input mode is used to input conversation content, and the second input mode is used to input non-conversation content.
[0091] In some embodiments, the non-dialogue content includes at least one of text, emoticons, and images.
[0092] In some embodiments, the output module 403 is further configured to: in the audio content, the content corresponding to the second input mode is identified by using a designated symbol, and the content corresponding to the first input mode is not identified by using a symbol.
[0093] In some embodiments, the audio input device 40 further includes a display module 404 configured to display audio input controls, wherein the styles of the audio input controls are different in different input modes.
[0094] It should be noted that the above-mentioned units are only logical modules divided according to the specific functions implemented by them, and are not used to limit the specific implementation methods. For example, they can be implemented in software, hardware, or a combination of software and hardware. In actual implementation, the above-mentioned units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (CPU or DSP, etc.), an integrated circuit, etc.). In addition, the above-mentioned units are shown with dotted lines in the drawings to indicate that these units may not actually exist, and the operations / functions implemented by them can be implemented by the processing circuit itself.
[0095] In addition, although not shown, the device may also include a memory, which may store various information generated by the device, various units contained in the device during operation, programs and data for operation, data to be sent by the communication unit, etc. The memory may be a volatile memory and / or a non-volatile memory. For example, the memory may include, but is not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-only memory (ROM), and a flash memory. Of course, the memory may also be located outside the device. Optionally, although not shown, the device may also include a communication unit, which may be used to communicate with other devices. In one example, the communication unit may be implemented in a suitable manner known in the art, such as including communication components such as an antenna array and / or a radio frequency link, various types of interfaces, communication units, etc. It will not be described in detail here. In addition, the device may also include other components not shown, such as a radio frequency link, a baseband processing unit, a network interface, a processor, a controller, etc. It will not be described in detail here.
[0096] Some embodiments of the present disclosure also provide an electronic device. Figure 5 A schematic diagram of the structure of an electronic device according to some embodiments of the present disclosure is shown. For example, in some embodiments, the electronic device 5 can be various types of devices, for example, can include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. For example, the electronic device 5 can include a display panel for displaying data and / or execution results used in the scheme according to the present disclosure. For example, the display panel can be in various shapes, such as a rectangular panel, an elliptical panel, or a polygonal panel, etc. In addition, the display panel can be not only a flat panel, but also a curved panel, or even a spherical panel.
[0097] like Figure 5 As shown, the electronic device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51. It should be noted that Figure 5 The components of the electronic device 5 shown are only exemplary and non-restrictive. The electronic device 5 may also have other components according to actual application requirements. The processor 52 may control other components in the electronic device 5 to perform desired functions.
[0098] In some embodiments, the memory 51 is used to store one or more computer-readable instructions. When the processor 52 is used to execute the computer-readable instructions, the computer-readable instructions are executed by the processor 52 to implement the method according to any of the above embodiments. For the specific implementation of each step of the method and the related explanation content, please refer to the above embodiments, and the repeated parts will not be repeated here.
[0099] For example, the processor 52 and the memory 51 may communicate with each other directly or indirectly. For example, the processor 52 and the memory 51 may communicate with each other through a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 52 and the memory 51 may also communicate with each other through a system bus, which is not limited by the present disclosure.
[0100] For example, the processor 52 may be embodied as various appropriate processors, processing devices, etc., such as a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The central processing unit (CPU) may be an X86 or ARM architecture, etc. For example, the memory 51 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The memory 51 may include, for example, a system memory, which may store, for example, an operating system, an application, a boot loader (Boot Loader), a database, and other programs. Various applications and various data may also be stored in the storage medium.
[0101] In addition, according to some embodiments of the present disclosure, various operations / processes according to the present disclosure, when implemented by software and / or firmware, can be transmitted from a storage medium or a network to a computer system having a dedicated hardware structure, such as Figure 6 The computer system 60 shown installs the programs constituting the software, and when the various programs are installed, the computer system can perform various functions, including the functions described above. Figure 6 A schematic diagram of the structure of a computer system according to some embodiments of the present disclosure is shown.
[0102] exist Figure 6In the embodiment, the central processing unit (CPU) 601 performs various processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. In the RAM 603, data required when the CPU 601 performs various processes, etc. are also stored as needed. The central processing unit is only exemplary, and it can also be other types of processors, such as the various processors described above. The ROM 602, RAM 603 and the storage part 608 can be various forms of computer-readable storage media, as described below. It should be noted that although Figure 6 ROM 602, RAM 603 and storage device 608 are shown separately in FIG. 6 , but one or more of them may be combined or located in the same or different memory or storage modules.
[0103] The CPU 601, the ROM 602, and the RAM 603 are connected to one another via a bus 604. An input / output interface 605 is also connected to the bus 604.
[0104] The following components are connected to the input / output interface 605: an input portion 606, such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output portion 607, including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage portion 608, including a hard disk, a magnetic tape, etc.; and a communication portion 609, including a network interface card such as a LAN card, a modem, etc. The communication portion 609 allows communication processing to be performed via a network such as the Internet. It is easy to understand that although Figure 6 The various devices or modules in the computer system 60 are shown to communicate via a bus 604, but they may also communicate via a network or other means, wherein the network may include a wireless network, a wired network, and / or any combination of wireless networks and wired networks.
[0105] A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.
[0106] In the case where the above-described series of processing is implemented by software, a program constituting the software may be installed from a network such as the Internet or a storage medium such as the removable medium 611 .
[0107] According to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the CPU 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0108] It should be noted that in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0109] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0110] In some embodiments, a computer program is further provided, comprising: instructions, which, when executed by a processor, cause the processor to execute any of the methods of the above embodiments. For example, the instructions may be embodied as computer program codes.
[0111] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In situations involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0113] The modules, components or units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the modules, components or units do not, in some cases, limit the modules, components or units themselves.
[0114] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0115] The above descriptions are only some embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, a technical solution formed by replacing the above features with the technical features with similar functions disclosed in the present disclosure (but not limited to).
[0116] In the description provided herein, many specific details are set forth. However, it is understood that embodiments of the present invention can be implemented without these specific details. In other cases, well-known methods, structures, and techniques are not presented in detail in order not to obscure the understanding of the description.
[0117] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0118] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.
Claims
1. An audio input method, comprising: In response to a first operation of the user on the audio input control, collecting audio; During the process in which the user performs a first operation on the audio input control, determining an input mode according to a second operation of the user, wherein different input modes correspond to different output modes; Content corresponding to the audio is output according to an output mode corresponding to the input mode.
2. The audio input method according to claim 1, wherein: The first operation is a long press operation.
3. The audio input method according to claim 1 or 2, wherein: The second operation includes a sliding gesture operation.
4. The audio input method according to claim 3, wherein: The second operation includes a sliding gesture operation along a specified direction.
5. The audio input method according to claim 1, further comprising: An initial input mode is determined according to a triggering area of the first operation performed by the user in the audio input control.
6. The audio input method according to claim 1, wherein: In the process in which the user performs the first operation on the audio input control, determining the input mode according to the second operation of the user includes: During the process in which the user performs a first operation on the audio input control, in response to the second operation performed by the user being a designated operation, the current input mode is switched to another input mode.
7. The audio input method according to claim 6, wherein: In response to the second operation performed by the user being a designated operation, switching the current input mode to another input mode includes: In response to the current input mode being the first input mode and the second operation performed by the user being the first designated operation, the current input mode is switched to the second input mode.
8. The audio input method according to claim 7, wherein: In response to the second operation performed by the user being a designated operation, switching the current input mode to another input mode further includes: In response to the current input mode being the second input mode and the second operation performed by the user being a second designated operation, the current input mode is switched to the first input mode.
9. The audio input method according to claim 8, wherein: The first designated operation and the second designated operation are sliding gesture operations with different sliding directions.
10. The audio input method according to claim 1, wherein: In the process in which the user performs the first operation on the audio input control, determining the input mode according to the second operation of the user includes: In the process of the user performing a first operation on the audio input control, dividing the audio or content corresponding to the audio into a plurality of sections according to the execution time of the user's second operation; An input mode for each paragraph is determined according to the input mode corresponding to the second operation of the user.
11. The audio input method according to claim 10, wherein: The dividing the audio or the content corresponding to the audio into a plurality of sections according to the execution time of the second operation of the user includes: Dividing the audio or content corresponding to the audio into a plurality of units according to the semantics of the audio; The plurality of units are divided into a plurality of paragraphs according to the execution time of the second operation of the user.
12. The audio input method according to claim 10 or 11, wherein: Outputting content corresponding to the audio according to the output mode corresponding to the input mode includes: According to an output mode corresponding to an input mode of each paragraph, content corresponding to the audio is output.
13. The audio input method according to claim 1, wherein: The output mode includes a first output mode and a second output mode; The first output mode is used to directly display the text corresponding to the audio; The second output mode is used to display text corresponding to the audio and identified by a designated symbol, or to display non-text information associated with the text corresponding to the audio.
14. The audio input method according to claim 1 or 13, wherein: The input mode includes a first input mode and a second input mode, the first input mode is used to input dialogue content, and the second input mode is used to input non-dialogue content.
15. The audio input method according to claim 14, wherein: The non-dialogue content includes at least one of text, expression, and image.
16. The audio input method according to claim 14, wherein: Outputting content corresponding to the audio according to the output mode corresponding to the input mode includes: In the audio content, the content corresponding to the second input mode is identified by using a designated symbol, and the content corresponding to the first input mode is not identified by using the symbol.
17. The audio input method according to claim 1, further comprising: The audio input control is displayed, wherein the style of the audio input control is different in different input modes.
18. An electronic device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the audio input method according to any one of claims 1 to 17 based on instructions stored in the memory.
19. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the audio input method according to any one of claims 1 to 17 is implemented.
20. A computer program product, when the computer program product is run on a computer, enables the computer to implement the audio input method according to any one of claims 1 to 17.
Citation Information
Patent Citations
Voice search method and apparatus
CN105095509A
Information inputting method and device
CN106531164A
Multi-modal interaction method, device and system based on virtual character, storage medium and terminal
CN112162628A
Data processing method, equipment and medium
CN116456160A
Interaction method and device, computer readable storage medium and computer program product
CN117959703A
Cited By
Audio input method, electronic device, storage medium, and computer program product
WO2026020850A1
