Method, device and terminal device for audio playback of text

Abstract

The present application provides a method, device and terminal device for audible playback of text, and relates to the field of terminal technology. The method includes: identifying non-text information in a target text; determining the audio information corresponding to the non-text information; and audibly playing the target text according to the audio information corresponding to the non-text information. The method for audible playback of text provided in this embodiment can fully express the non-text information in the target text during the process of audibly playing the target text, improve the expression effect of the terminal device on the text information, and improve the user experience.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method, device and terminal device for audible playback of text. Background Art

[0002] Terminal devices usually have the function of reading text aloud. At present, the way for terminal devices to read text aloud is to play the pronunciation audio corresponding to the text in the text, so as to achieve the purpose of reading the text aloud. However, the text read aloud by the terminal device includes not only text, but also some non-text information, such as punctuation marks, emoticons, underlined fonts, etc., which are used to express the sentence reading, tone, emotion, etc. of the text, or highlight some text. Since the current terminal devices only read the text in the text when reading aloud, therefore, for texts including non-text information, the non-text information in the text cannot be fully expressed, resulting in poor expression of the text information. Summary of the invention

[0003] The present application provides a method, device and terminal device for audible playback of text, which improves the problem in the prior art that the terminal device has a poor expression effect on text information when reading the text aloud.

[0004] In order to achieve the above purpose, this application adopts the following technical solutions:

[0005] In a first aspect, the present application provides a method for audibly playing a text, comprising: identifying non-text information in a target text; determining audio information corresponding to the non-text information; and audibly playing the target text according to the audio information corresponding to the non-text information.

[0006] The audio playback method of text provided in this embodiment can play the audio information corresponding to the non-text information in the target text during the audio playback process, thereby fully expressing the non-text information in the text, improving the expression effect of the text information, and improving the user experience.

[0007] In combination with the first aspect, in some embodiments, the non-text information includes emoticons, typesetting control symbols, punctuation marks, mathematical symbols, annotation symbols, comment symbols, or characteristic font styles of text.

[0008] In combination with the first aspect, in some embodiments, when the target text includes text and the non-text information, determining the audio information corresponding to the non-text information includes:

[0009] If the non-text information is a first symbol, then determining the application type of the first symbol according to the semantics of the target text; the first symbol includes at least two audio information corresponding to the application type;

[0010] Audio information of the first symbol is determined according to the application type of the first symbol.

[0011] For first symbols that have different readings in different language scenarios, the audio information corresponding to the first symbol and the scenario can be determined by identifying the semantics of the target text, thereby preventing the terminal device from playing audio information that is not suitable for the scenario and improving the expression effect of the text information.

[0012] In combination with the first aspect, in some embodiments, when the target text includes text and the non-text information, playing the target text audibly according to the audio information corresponding to the non-text information includes:

[0013] If the non-text information is the emoticon, the typesetting control symbol, the punctuation mark, the mathematical symbol or the annotation symbol, the audio information corresponding to the text and the audio information corresponding to the non-text information are played in sequence according to the arrangement order of the text and the non-text information in the target text.

[0014] In combination with the first aspect, in some embodiments, when the target text includes text and the non-text information, playing the target text audibly according to the audio information corresponding to the non-text information includes:

[0015] If the non-text information is the annotation symbol, identifying the annotation text corresponding to the annotation symbol;

[0016] According to the arrangement order of the characters and the annotation symbols in the target text, the audio information corresponding to the characters and the audio information corresponding to the annotated characters are played in sequence.

[0017] In combination with the first aspect, in some embodiments, when the target text includes text and the non-text information, playing the target text audibly according to the audio information corresponding to the non-text information includes:

[0018] If the non-text information is the annotation symbol, identifying the annotation text corresponding to the annotation symbol;

[0019] After playing the audio information of all the texts in the sentence where the non-text information is located, play the audio information corresponding to the annotation text.

[0020] By playing the audio information corresponding to the annotation text during the process of playing the audio information of the text, the user can understand the text information in more detail.

[0021] In combination with the first aspect, in some embodiments, when the target text includes text and non-text information, playing the target text audibly according to audio information corresponding to the non-text information includes:

[0022] If the non-text information is in the characteristic font style, then while playing the audio information corresponding to the text with the characteristic font style, the audio information corresponding to the characteristic font style is played as the background sound of the audio information corresponding to the text with the characteristic font style.

[0023] In combination with the first aspect, in some embodiments, determining the audio information corresponding to the non-text information includes:

[0024] According to the identification information of the non-text information, the audio information corresponding to the non-text information is determined from a preset audio information library.

[0025] In a second aspect, the present application provides a text audio playback device, the device comprising:

[0026] Recognition unit, used to recognize non-text information in the target text;

[0027] A determination unit, configured to determine audio information corresponding to the non-text information;

[0028] A playing control unit is used to play the target text audibly according to the audio information corresponding to the non-text information.

[0029] In a third aspect, this embodiment provides a terminal device, including a speaker, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for audible playback of text as described in the first aspect above is implemented.

[0030] In a fourth aspect, the present embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for audio playback of text as described in the first aspect above is implemented.

[0031] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the method for audible playback of text described in the first aspect above.

[0032] In a sixth aspect, an embodiment of the present application provides a chip system, the chip system comprising a processor, the processor coupled to a memory, the processor executing a computer program stored in the memory to implement the method for audio playback of text as described in the first aspect above. In this embodiment, the chip system can be a single chip, or a chip module composed of multiple chips.

[0033] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the structure of a mobile phone to which the method for audio playback of text provided in an embodiment of the present application is applicable;

[0035] Figure 2 A schematic diagram of the structure of a processor to which the method for playing text with voice provided in an embodiment of the present application is applicable;

[0036] Figure 3 A schematic diagram of a software architecture applicable to the audio playback method of text provided in an embodiment of the present application;

[0037] Figure 4 A flowchart of a method for playing text with sound provided in an embodiment of the present application Figure 1 ;

[0038] Figure 5a A user control diagram of audio playback of text provided in an embodiment of the present application Figure 1 ;

[0039] Figure 5b A user control diagram of audio playback of text provided in an embodiment of the present application Figure 2 ;

[0040] Figure 6a An emoji emoticon provided in an embodiment of the present application Figure 1 ;

[0041] Figure 6b An emoji emoticon provided in an embodiment of the present application Figure 2 ;

[0042] Figure 6c An emoji emoticon provided in an embodiment of the present application Figure 3 ;

[0043] Figure 6d An emoji emoticon provided in an embodiment of the present application Figure 4 ;

[0044] Figure 7 A display diagram of a program code provided in an embodiment of the present application Figure 1 ;

[0045] Figure 8 A display diagram of a program code provided in an embodiment of the present application Figure 2 ;

[0046] Fig. 9 A flowchart of a method for playing text with sound provided in an embodiment of the present application Figure 2 ;

[0047] Fig.10 A schematic diagram of the structure of a text audio playback device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The audio playback method of text provided in the embodiment of the present application can be applied to terminal devices with audio information playback function such as mobile phones, tablet computers, e-readers, laptops, netbooks, wearable devices, etc. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.

[0049] Take the terminal device as an example, which is a mobile phone. Figure 1 FIG. 1 is a block diagram of a partial structure of a mobile phone provided in an embodiment of the present application. Figure 1 The mobile phone includes components such as a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a wireless fidelity (WiFi) module 170, a processor 180, and a power supply 190. Those skilled in the art will appreciate that Figure 1 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0050] Combine the following Figure 1 A detailed introduction to the various components of the mobile phone:

[0051] The RF circuit 110 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 180 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 110 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE)), email, Short Messaging Service (SMS), etc.

[0052] The memory 120 can be used to store software programs and modules. The processor 180 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 120. The memory 120 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as an audio information playback function, a text display function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio information, a phone book, etc.), etc. In addition, the memory 120 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0053] The input unit 130 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 130 may include a touch panel 131 and other input devices 132. The touch panel 131, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 131 or near the touch panel 131 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 180, and can receive and execute commands sent by the processor 180. In addition, the touch panel 131 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 131, the input unit 130 may also include other input devices 132. Specifically, other input devices 132 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0054] The display unit 140 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 140 may include a display panel 141. Optionally, the display panel 141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 131 may cover the display panel 141. When the touch panel 131 detects a touch operation on or near it, it is transmitted to the processor 180 to determine the type of touch event. Subsequently, the processor 180 provides a corresponding visual output on the display panel 141 according to the type of touch event. Although in Figure 1 In the embodiment, the touch panel 131 and the display panel 141 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 131 and the display panel 141 can be integrated to realize the input and output functions of the mobile phone.

[0055] The mobile phone may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 141 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. As for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0056] The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the mobile phone. The audio circuit 160 can transmit the received audio data to the speaker 161 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the processor 180 for processing, and then sent to another mobile phone through the RF circuit 110, or the audio data is output to the memory 120 for further processing.

[0057] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 170, which provides users with wireless broadband Internet access. Figure 1 A WiFi module 170 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0058] The processor 180 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 120, and calling data stored in the memory 120, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 180 may include one or more processing units; preferably, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 180.

[0059] See also Figure 2, the processor 180 may include: a text control unit (Text View) 1801, a text drawing unit (Draw Text) 1802, a text rendering engine 1803, a read-aloud trigger 1804 and a text-to-speech engine (Text toSpeech, TTS) 1805. Among them, the text control unit 1801 is used to determine the display information of the text, such as the content, font size, and display shape. The text drawing unit 1802 is used to control the typesetting style of the text. The text rendering engine 1803 is used to finally determine the display image of the text in the display interface according to the display information and typesetting style of the text. The read-aloud trigger 1804 is used to select the text according to the user's touch operation, determine it as the target text, and control the mobile phone to start playing the target text in sound. The text-to-speech engine 1805 is used to cooperate with the speaker 161 to play the audio information corresponding to the target text.

[0060] The mobile phone also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 180 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0061] Although not shown, the mobile phone may further include a camera. Optionally, the position of the camera on the mobile phone may be front-mounted or rear-mounted, which is not limited in the embodiments of the present application. Optionally, the mobile phone may include a single camera, a dual camera, or a triple camera, etc., which is not limited in the embodiments of the present application. For example, the mobile phone may include three cameras, one of which is a main camera, one is a wide-angle camera, and one is a telephoto camera. When the mobile phone includes multiple cameras, these multiple cameras may all be front-mounted, or all be rear-mounted, or some may be front-mounted and others may be rear-mounted, which is not limited in the embodiments of the present application.

[0062] In addition, although not shown, the mobile phone may also include a Bluetooth module, etc., which will not be described in detail here.

[0063] Figure 3 Schematic diagram of the software structure of the mobile phone of the embodiment of the present application. Taking the mobile phone operating system as the Android system as an example, in some embodiments, the Android system is divided into four layers, namely the application layer, the application framework layer (framework, FWK), the system layer and the hardware abstraction layer, and the layers communicate with each other through software interfaces.

[0064] like Figure 3 As shown, the application layer can be a series of application packages, and the application packages can include short message, calendar, camera, video, navigation, gallery, call and other applications.

[0065] The application framework layer provides an application programming interface (API) and a programming framework for the application programs of the application layer. The application framework layer may include some predefined functions, such as a function for receiving events sent by the application framework layer.

[0066] like Figure 3 As shown, the application framework layer may include a window manager, a resource manager, a notification manager, and the like.

[0067] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0068] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0069] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0070] The application framework layer may also include:

[0071] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying pictures.

[0072] The phone manager is used to provide the communication functions of the mobile phone, such as the management of call status (including answering, hanging up, etc.).

[0073] The system layer may include multiple functional modules, such as sensor service module, physical state recognition module, 3D graphics processing library (such as OpenGL ES), etc.

[0074] The sensor service module is used to monitor the sensor data uploaded by various sensors at the hardware layer and determine the physical state of the phone;

[0075] Physical state recognition module, used to analyze and recognize user gestures, faces, etc.;

[0076] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0077] The system layer may also include:

[0078] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0079] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0080] The hardware abstraction layer is a layer between hardware and software. The hardware abstraction layer can include display drivers, camera drivers, sensor drivers, etc., which are used to drive related hardware of the hardware layer, such as display screens, cameras, sensors, etc.

[0081] The following embodiments can be implemented on a terminal device having the above hardware structure / software structure. The following embodiments will take a mobile phone as an example to illustrate the audio playback method of text provided in the embodiments of the present application.

[0082] Mobile phones usually have the function of reading text aloud. At present, the way mobile phones read text aloud is to only play the pronunciation audio corresponding to the text in the text, so as to achieve the purpose of reading the text aloud. However, these texts may not only include text, but also some non-text information, such as punctuation marks, emoticons, underlines, etc., which are used to express the sentence reading, tone or emotion of the text, or highlight some text. Therefore, for texts including non-text information, when the mobile phone adopts the above-mentioned text reading method, it cannot fully express the non-text information in the text, affecting the expression effect of the text information.

[0083] For example, for the text "Nice to meet you ^_^", the current way of reading the text by mobile phones is to only play the audio information of the pronunciation of all the words "Nice to meet you". However, the non-text information "^_^" in the text is not expressed, which affects the expression effect of the text information.

[0084] To this end, an embodiment of the present application provides a method for audio playback of text, which can express non-text information in the text and improve the expression effect of the text information.

[0085] See also Figure 4, A schematic flowchart of a method for voice-playing text provided by an embodiment of the present application. The method includes the following steps S401-S403.

[0086] S401, The mobile phone recognizes non-text information in the target text.

[0087] In this embodiment, the target text refers to the text selected by the user for voice-playing by the mobile phone, which may include all the content in a text (such as a novel), or a part of the text (such as some paragraphs in the novel), and this embodiment does not limit it.

[0088] In some embodiments, as shown in Figure 5a , when the mobile phone displays the text, the user can select a part of the text in the text as the target text by touching, and click the play icon to input a play instruction to the mobile phone to control the mobile phone to voice-play the target text. In Figure 5a , the text selected by the user is shown in shadow. In other embodiments, as shown in Figure 5b , the user can also directly click the play icon to control the mobile phone to use the entire text as the target text and voice-play the text.

[0089] In the target text stored in the mobile phone, identification information of each character and non-text information therein is included. Among them, the identification information is used to uniquely indicate a character or non-text information. Through the identification information of the target text, the mobile phone can recognize the characters and non-text information in the text.

[0090] Exemplarily, the identification information may be a character encoding, such as Unicode. For example, the 16-bit Unicode of "王" is U+738B, the 16-bit Unicode of "," is U+002C, the 16-bit Unicode of the tab character is U+0009, the 16-bit Unicode of the underline is U+2381, etc.

[0091] It should be noted that the characters involved in this embodiment may be in various language forms such as Chinese, English, Japanese, French, etc., for example, they may be "高", "兴", "happy", "お", "Bonjour", etc. The non-text information may be emoticons, punctuation marks, mathematical symbols, typesetting control symbols, annotation symbols, comment symbols, or the characteristic font styles of characters, etc.

[0092] Among them, emoticons are usually used to represent a certain emotion, such as a happy emotion, a crying emotion, a bored emotion, a cheering emotion, etc. However, in this embodiment, the emoticons are not limited to this, and can also be used to represent a certain thing, such as the moon, Christmas tree, house, flower, etc.

[0093] In some embodiments, emoticons may be composed of some non-text symbols, or may be composed of non-text symbols and text. For example, these non-text symbols may be “^”, “_”, “(”, “﹏”, “)”, “{”, “→”, “>”, “<”, “|”, “\”, and “ / ”, etc. The combined emoticons may be “^_^”, “(>﹏<)”, “>_<|||”, Y(^_^)Y, etc. Each emoticon is used to represent an expression, as shown in Table 1.

[0094] Table 1 Correspondence Table 1

[0095] In some other embodiments, the emoticon may be an emoji. Figure 6a The emoji symbol shown is used to represent a smiling face. Figure 6b An emoji is shown representing a tearful face. Figure 6c The emoji symbol shown is used to represent an unhappy expression. Figure 6d The emoji shown is used to represent a frightened expression.

[0096] In this embodiment, punctuation marks include ",",".",";,",",":,"!"!","!"!","(,")","{,"}","

","

[0097] In this embodiment, mathematical symbols include "+", "-", "÷", " / ", "log", "m (meter)", "mm (millimeter)", ": (ratio)", "℃" and other symbols used for data calculations or to represent units.

[0098] Typesetting control symbols may be tab symbols, line breaks, section breaks, etc., which are symbols used to control the typesetting of text, punctuation marks, emoticons, etc. in the display interface, so as to clearly express the hierarchy of the text and facilitate user reading. According to the user's settings, these typesetting control symbols can be displayed in the display interface or hidden in the display interface. Exemplarily, when a tab symbol is displayed in the display interface, it can be represented by "→". When a line break is displayed in the display interface, it can be represented by "CRLF". When a space is displayed in the display interface, it can be represented by a "·" that is lighter in color than the text.

[0099] The annotation symbol can be a superscript or subscript of a character or word, corresponding to the annotation text regarding that character or word. For example, in the text "I wonder what year it is in the celestial palace① tonight", the annotation symbol is "①", and its corresponding annotation text is "The celestial palace refers to the palace. Since there are double watchtowers outside the palace gate, it is called the celestial palace."

[0100] The annotation symbol can be a symbol used in program code to represent an explanation. For example, it can include the first annotation symbol " / *", the second annotation symbol "* / ", and the third annotation symbol " / / ". In program code, " / *" and "* / " are used in combination, and the text between " / *" and "* / " is the annotation text, which can occupy multiple lines. " / / " is usually used alone, and the text after " / / " is the annotation text, which usually occupies only one line.

[0101] The characteristic font style can be italic, underlined font, strikethrough font, bold font, font with a background color, or colored font, etc. For example, in the text "You should ensure that all information" Accurate ", the characteristic font style is the underlined font.

[0102] S402, the mobile phone determines the audio information corresponding to the non-text information.

[0103] There is a preset audio information library in the mobile phone, which includes the audio information corresponding to text and non-text information. Exemplarily, the correspondence between text or non-text information and audio information can be shown in Table 2.

[0104] Table 2 Correspondence Table Two

[0105]

[0106] For text, its audio information usually corresponds to the pronunciation of that text. For example, the audio information corresponding to the text "You" has the pronunciation of [You]. The audio information corresponding to the text "Good" has the pronunciation of [Good].

[0107] For non-text information, the emotion expressed by the audio information corresponding to the emoji should be the same as the emotion expressed by that emoji. For example, for the emoji expressing a big smile its corresponding audio information can have the pronunciation of [Hahaha]. The tone expressed by the audio information of punctuation marks should be the same as the tone expressed by that punctuation mark. For example, for the punctuation mark "?" indicating a question, it can correspond to the audio information with the pronunciation of [Eh]. The audio information corresponding to the characteristic font style should enable the user to clearly understand the information conveyed by that characteristic font style. For example, the audio information corresponding to the underlined font can be the sound of a pencil drawing a line. In this embodiment, the specific content of the audio information corresponding to non-text information is not limited.

[0108] It should be noted that in the audio information library, each piece of text or non-text information corresponds to at least one audio information. For example, the audio information of the text "您" can be a female voice pronounced as

您

您

或

除以

[0109] In addition, for punctuation marks such as ",", ".", ";", "、" etc. that appear too frequently and are used to indicate sentence readings, corresponding audio information may not be set in the audio information library to avoid inserting audio information too frequently during the audible playback of the text, resulting in a reduced user experience.

[0110] S403, the mobile phone audibly plays the target text according to the audio information corresponding to the non-text information.

[0111] In the target text, both text and non-text information have definite position information, which is used to represent the arrangement order of the text and non-text information in the target text. For example, for a character (including text, punctuation marks, mathematical symbols, emoji, annotation symbols, comment symbols, etc.), when its position information is 5, it means that the character is the 5th character in the target text. For a characteristic font style (such as bold font, italic or underline, etc.), when its position information is from 10 to 15, it means that the characteristic font style is at the position where the 10th to 15th characters are located.

[0112] For the sake of convenience in description, hereinafter, the position information from a to b will be uniformly represented as [a, b], where a is less than or equal to b, and both a and b are integers. For example, in this embodiment, the position information 5 is represented as [5, 5], and the position information from 10 to 15 is represented as [10, 15].

[0113] It should be noted that in this embodiment, a character that is displayed or pronounced as a whole corresponds to a display position, and each display position corresponds to a position information. For example, a Chinese character (such as "您", "好"), an English word (such as "happy", "a", etc.), a punctuation mark (such as ",", "."), an emoji (such as "^_^" or an emoji expression), an annotation symbol (such as " ① ”、“ ②”), a typesetting control symbol (such as a space, a tab or a line break), etc., each corresponds to a display position and has a corresponding position information. Of course, the method for determining the position information can also be in other forms, which is not limited in this embodiment.

[0114] Taking the target text "Hello, Teacher Wang, ^_^ Nice to meet you" as an example, the position information of the text "Hello, Teacher Wang" in the target text is [1,5], the position information of the punctuation mark "," in the target text is [6,6], the position information of the emoticon "^_^" in the target text is [7,7], and the position information of "Nice to meet you" in the target text is [8,13] respectively.

[0115] With the target text "You should ensure that the information Accurate " is taken as an example, where the position information of the underline font in the target text is [8,11].

[0116] With the target text "I don't know the palace in heaven ① , what year is this evening? ① The corresponding annotation text is "Palace refers to the palace, because there are two palaces outside the palace gate, so it is called palace." Among them, the annotation symbol ① The position information in the target text is [7,7].

[0117] In some embodiments, the target text only includes non-text information. Then, when the mobile phone plays the target text with sound, it can directly play the audio information corresponding to the non-text information. For example, in a chat scenario, the target text only includes a Figure 6a Then, when the mobile phone plays the target text, it can directly play the audio information corresponding to the emoji emoticon.

[0118] In other embodiments, the target text includes text and non-text information. Then the mobile phone can have different playing modes when playing the target text. The following will take a specific text as an example to illustrate the playing mode of the target text provided by the present application.

[0119] Taking the target text "Hello, Mr. Wang, ^_^ Nice to meet you" as an example, when the mobile phone plays the target text with sound, it can play the audio information corresponding to the text and the audio information corresponding to the non-text information in sequence according to the arrangement order of the text and non-text information in the target text. The details are as follows.

[0120] First, identify that the position information of "Hello, Teacher Wang" in the target text is [1, 5], the punctuation mark ",", in the target text is [6, 6], the emoticon "^_^" in the target text is [7, 7], and the position information of "Nice to meet you" in the target text is [8, 13] in sequence. Subsequently, the mobile phone obtains the audio information corresponding to the characters "王 (Wang)", "老 (Lao)", "师 (Shi)", "您 (You)", "好 (Hao)", "很 (Hen)", "高 (Gao)", "兴 (Xing)", "认 (Ren)", "识 (Shi)", and "^_^" from the preset audio information library. Finally, the mobile phone plays the audio information of "王 (Wang)" first, the audio information of "老 (Lao)" second, the audio information of "师 (Shi)" third, the audio information of "您 (You)" fourth, the audio information of "好 (Hao)" fifth, pauses for a preset time (such as 0.5s) at the sixth playback position (i.e., the position corresponding to ","), to indicate the pause of the sentence, plays the audio information of "^_^" (

哈哈哈哈

[0121] It should be noted that, for example, in a chat scenario, when the target text includes an emoticon, the audio information corresponding to the emoticon will be added to the text sound played by the mobile phone, making the expression of the text information more vivid and vivid.

[0122] Taking the target text "您应确保信息都 Accurate " as an example, during the process of the mobile phone audibly playing this target text, it can play the audio information corresponding to the underscore simultaneously during the process of playing the audio information of the text "准确无误 (accurate and error-free)", as the background sound of the audio information of "准确无误 (accurate and error-free)". Specifically as follows.

[0123] First, the mobile phone identifies that the position information of "您应确保信息都准确无误 (You should ensure that the information is accurate and error-free)" in the target text is [1, 11] in sequence, and the position information of the underscore is [8, 11]. Subsequently, the mobile phone obtains the audio information of "您 (You)", "应 (should)", "确 (ensure)", "保 (guarantee)", "信 (information)", "息 (message)", "都 (all)", "准 (accurate)", "确 (correct)", "无 (without)", "误 (error)" respectively from the preset audio information library, as well as the audio information of the underscore. Finally, play the audio information of "您 (You)" first according to the above position information, play the audio information of "应 (should)" second, and so on, play the audio information of "确 (ensure)", "保 (guarantee)", "信 (information)", "息 (message)", "都 (all)", "准 (accurate)", "确 (correct)", "无 (without)", "误 (error)". And when the mobile phone starts to play the audio information of "准 (accurate)", it starts to play the audio information of the underscore (such as the sound of a pencil drawing a line), until the audio information of "误 (error)" is played. That is to say, during the process of playing the audio information of the text "准 (accurate)", "确 (correct)", "无 (without)", "误 (error)", play the audio information corresponding to the underscore.

[0124] Taking the target text "I wonder what year it is in the celestial palace ① , tonight" as an example, during the process of the mobile phone playing the target text audibly, it can insert the annotation symbol ① corresponding to the audio information of the annotation text "Palace refers to the palace. Since there are double towers outside the palace gate, it is called palace". The specific situation is as follows.

[0125] First, the mobile phone identifies that the position information of "I wonder what year it is in the celestial palace" in the target text is [1, 6], the position information of the annotation symbol " ① " in the text is [7, 7], the position information of "," is "[8, 8]", and the position information of "tonight" is [9, 13]. Subsequently, the mobile phone respectively obtains the audio information of each character in "I wonder what year it is in the celestial palace, tonight" and the audio information of each character corresponding to the annotation text "Palace refers to the palace. Since there are double towers outside the palace gate, it is called palace" from the preset audio information library. Finally, the mobile phone combines the audio information of the annotation text corresponding to the annotation symbol " ① " and plays the target text.

[0126] In a possible implementation manner, the mobile phone can play the audio information corresponding to the characters and the audio information corresponding to the annotation text in the order of the arrangement of the characters and the annotation symbols in the target text. For example, it plays the audio information of "I wonder what year it is in the celestial palace" in sequence, then plays the audio information of the annotation text "Palace refers to the palace. Since there are double towers outside the palace gate, it is called palace", and finally plays the audio information corresponding to "tonight".

[0127] In another possible implementation manner, the mobile phone can play the audio information corresponding to the annotation text after playing the audio information of all the characters in the sentence where the non-character information is located. For example, the mobile phone can play the audio information of each character corresponding to "I wonder what year it is in the celestial palace, tonight" in sequence, and then play the audio information of each character corresponding to the annotation text "Palace refers to the palace. Since there are double towers outside the palace gate, it is called palace".

[0128] In this embodiment, by playing the audio information corresponding to the annotation text during the process of playing the audio information of the characters, it can enable the user to understand the text information in more detail.

[0129] Taking the target text as the program code shown in Figure 7 for example, this program code includes multiple layout control symbols. During the process of the mobile phone playing the target text audibly, it can insert the audio information corresponding to the layout control symbols during the process of playing the audio information of the program code characters.

[0130] It should be noted that, usually, when displaying program code, the mobile phone does not display typesetting control symbols to avoid affecting the user's reading experience. However, for the convenience of describing this embodiment, for example Figure 7 As shown, this embodiment shows program code displayed with typesetting control symbols.

[0131] When the mobile phone plays the seventh line of the program code with sound, it first recognizes that the position information of the tab symbol, "dependences", space symbol, "{", and "CRLF" included therein are 1, 2, 3, 4, and 5 respectively. Subsequently, the mobile phone obtains the audio information of the tab symbol, "dependences", space symbol, "{", and carriage return symbol from the preset audio information library, and plays the audio information of the tab symbol, "dependences", space symbol, "{", and carriage return symbol in sequence according to the above position information, thereby expressing the typesetting information of the text during the process of playing the text with sound.

[0132] In an example, the audio information of typesetting control symbols such as tab symbols, space symbols, and carriage return symbols may correspond to different keyboard tapping sounds, and this embodiment does not limit the specific content thereof.

[0133] It should be noted that in a text, a space is usually used only to separate two words or symbols. Therefore, although a space occupies a display position, the mobile phone may not play the corresponding audio information during the process of playing the target text with sound.

[0134] The target text is Figure 8 Take the program code shown as an example. There are multiple comments in the program code. The text from lines 4 to 6 is a comment, which includes the comment symbols " / *" and "* / ", and the comment text "The specific URL of jcenter needs to be specified here". Line 11 is a comment, which includes the comment symbol " / / " and the comment text "The version must be above 3.2.1"

[0135] In a possible implementation, when the terminal device encounters annotation information during the process of playing the program code, it may not play the audio information corresponding to the annotation symbol and annotation text. Figure 8 In the process of the program code shown, the audio information of the text in lines 4 to 6 and line 11 may not be played.

[0136] In another possible implementation, when the terminal device encounters annotation information during the process of playing the program code with sound, it can play the audio information corresponding to the annotation symbol and the annotation text in sequence according to the position information of the annotation symbol and the annotation text. The audio information corresponding to the annotation symbol can be ["ding dong"] or [code annotation], etc. It should be noted that ["ding dong"] is used to represent the sound effect of the audio information of the annotation symbol.

[0137] For example, the terminal device plays audio Figure 8 When the 4th to 6th lines of program code are played, they can be played as: [“Ding Dong” needs to specify the specific URL of jcenter here] or [Code comment needs to specify the specific URL of jcenter here]. Figure 8 In the process of the 11th line of text, it can be played as: [The "Ding Dong" version must be above 3.2.1] or [The code comment version must be above 3.2.1].

[0138] In addition, the non-text information of the target text may include some first symbols, which have different pronunciations in different language scenarios. For example, the first symbol may be " / ", and in the text "she / he", " / " is pronounced as [or]; while in the text "if a=100, b=20, please calculate the value of a / b", " / " is pronounced as [divide by].

[0139] In order to enable the mobile phone to accurately play the first symbol in the target text, see Fig. 9 This embodiment also provides a method for audio playback of text, including the following steps S901-S904.

[0140] S901, the mobile phone recognizes the first symbol in the target text.

[0141] In this embodiment, a first symbol list is maintained in the mobile phone, which includes identification information of multiple first symbols. For each character in the text, the mobile phone compares its identification information with the first symbol list. If the identification information of the character can be found in the first symbol list, the character is determined as the first character.

[0142] S902: The mobile phone determines the application type of the first symbol according to the semantics of the target text.

[0143] In this embodiment, the application type is used to indicate that a symbol is used as a punctuation mark, or as a mathematical symbol, or as an annotation symbol, etc. In addition, the first symbol includes audio information corresponding to at least two application types.

[0144] In some embodiments, the mobile phone can identify the semantics of the target text based on the keywords in the target text and determine the application type of the first symbol. For example, by identifying words such as "calculation", "value", "comparison", and "absolute value", it can be determined that the target text describes information related to mathematical operations, thereby determining that the type of the first symbol is a mathematical symbol. Taking the target text "If a=100, b=20, please calculate the value of a / b" as an example, the mobile phone can determine that the application type of " / " is a mathematical symbol based on the words "=", "calculation", and "value" therein.

[0145] S903: The mobile phone determines audio information corresponding to the first symbol according to the application type of the first symbol.

[0146] In the preset audio information library, the first symbol includes audio information corresponding to at least two application types. Taking the first symbol as " / " as an example, combined with Table 1, its audio information is audio information 8-1: [or], and audio information 8-2: [divided by].

[0147] For the target text "If a=100, b=20, please calculate the value of a / b", since the application type of " / " is a mathematical symbol, its corresponding audio information is audio information 8-2: [divide].

[0148] S904, the mobile phone plays the target text audibly according to the audio information corresponding to the first symbol.

[0149] Taking the target text "If a=100, b=20, please calculate the value of a / b" as an example, the mobile phone can identify the position information of each character in the process of playing the target text, and the position information of the punctuation mark " / " is [17,17]. In the process of playing the audio information of each character in sequence, the mobile phone plays the audio information 8-2 of " / ": [divided by] at the playback time corresponding to the 17th character. Therefore, the target text "If a=100, b=20, please calculate the value of a / b" is read as [If a equals one hundred and b equals twenty, please calculate the value of a divided by b], instead of being incorrectly read as [If a equals one hundred and b equals twenty, please calculate the value of a or b].

[0150] It should be noted that the above embodiments do not restrict the order in which the mobile phone obtains audio information from the audio information library and determines the location information. In other words, the mobile phone can first determine the location information and then obtain audio data; or it can first obtain audio data and then determine the location information.

[0151] To sum up, the audio playback method of text provided in this embodiment can play the audio information corresponding to the non-text information during the audio playback of the target text, thereby fully expressing the non-text information in the text, improving the expression effect of the terminal device on the text information, and improving the user experience.

[0152] Corresponding to the audio playback method of text shown in the above embodiment, this embodiment also provides an audio playback device for text. For the convenience of description, only the parts related to the embodiment of the present application are shown.

[0153] See also Fig.10 The text audio playback device provided in this embodiment includes a recognition unit 1001, a determination unit 1002 and a playback control unit 1003.

[0154] The recognition unit 1001 is used to recognize non-text information in the target text.

[0155] The determination unit 1002 is used to determine the audio information corresponding to the non-text information.

[0156] The play control unit 1003 is used to play the target text audibly according to the audio information corresponding to the non-text information.

[0157] Optionally, the non-text information includes emoticons, typesetting control symbols, punctuation marks, mathematical symbols, annotation symbols, or characteristic font styles of text.

[0158] Optionally, the determination unit 1002 is further configured to determine audio information corresponding to the non-text information from a preset audio information library according to identification information of the non-text information.

[0159] Optionally, when the target text includes text and non-text information, the determination unit 1002 is also used to determine the application type of the first symbol according to the semantics of the target text if the non-text information is a first symbol, wherein the first symbol includes audio information corresponding to at least two application types; and, according to the application type of the first symbol, determine the audio information of the first symbol.

[0160] Optionally, when the target text includes text and the non-text information, the playback control unit 1003 is also used to play the audio information corresponding to the text and the audio information corresponding to the non-text information in sequence according to the arrangement order of the text and the non-text information in the target text if the non-text information is emoticons, typesetting control symbols, punctuation marks or mathematical symbols.

[0161] Optionally, when the target text includes text and the non-text information, the playback control unit 1003 is also used to identify the annotation text corresponding to the annotation symbol if the non-text information is the annotation symbol; and play the audio information corresponding to the text and the audio information corresponding to the annotation text in sequence according to the arrangement order of the text and the annotation symbols in the target text.

[0162] Optionally, when the target text includes text and the non-text information, the playback control unit 1003 is also used to identify the annotation text corresponding to the annotation symbol if the non-text information is the annotation symbol; and, after playing the audio information of all the text in the sentence containing the non-text information, play the audio information corresponding to the annotation text.

[0163] Optionally, when the target text includes text and non-text information, the playback control unit 1003 is also used to, if the non-text information is in a characteristic font style, play the audio information corresponding to the characteristic font style at the same time as playing the audio information corresponding to the text with the characteristic font style, as background sound for the audio information corresponding to the text with the characteristic font style.

[0164] This embodiment also provides a terminal device, which includes a speaker, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for playing text with voice provided by this embodiment is implemented. Figure 1 shown.

[0165] This embodiment further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0166] The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the font audio playback device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electric carrier signal and a telecommunication signal.

[0167] The embodiment of the present application further provides a computer program product including instructions. When the computer program product is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0168] The embodiment of the present application provides a chip system, the chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the audio playback method of the text provided in the embodiment of the present application as described above. In this embodiment, the chip system can be a single chip, or a chip module composed of multiple chips.

[0169] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0170] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instruction can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instruction can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrations.

[0171] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0172] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM and other media that can store program codes.

[0173] Finally, it should be noted that the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for playing text with voice, characterized in that: include: Identify non-textual information in the target text; Determining audio information corresponding to the non-text information; Playing the target text audibly according to the audio information corresponding to the non-text information; Among them, if the non-text information is a characteristic font style of text, then while playing the audio information corresponding to the text with the characteristic font style, the audio information corresponding to the characteristic font style is played as the background sound of the audio information corresponding to the text with the characteristic font style.

2. The method according to claim 1, characterized in that The non-text information includes emoticons, typesetting control symbols, punctuation marks, mathematical symbols, annotation symbols, comment symbols or characteristic font styles of text.

3. The method according to claim 1, characterized in that: When the target text includes text and the non-text information, determining the audio information corresponding to the non-text information includes: If the non-text information is a first symbol, then determining the application type of the first symbol according to the semantics of the target text; the first symbol includes at least two audio information corresponding to the application type; Audio information of the first symbol is determined according to the application type of the first symbol.

4. The method according to claim 2, characterized in that: When the target text includes text and the non-text information, the target text is played audibly according to the audio information corresponding to the non-text information, including: If the non-text information is the emoticon, the typesetting control symbol, the punctuation mark, the mathematical symbol or the annotation symbol, the audio information corresponding to the text and the audio information corresponding to the non-text information are played in sequence according to the arrangement order of the text and the non-text information in the target text.

5. The method according to claim 2, characterized in that: When the target text includes text and the non-text information, the target text is played audibly according to the audio information corresponding to the non-text information, including: If the non-text information is the annotation symbol, identifying the annotation text corresponding to the annotation symbol; According to the arrangement order of the characters and the annotation symbols in the target text, the audio information corresponding to the characters and the audio information corresponding to the annotated characters are played in sequence.

6. The method according to claim 2, characterized in that When the target text includes text and the non-text information, the target text is played audibly according to the audio information corresponding to the non-text information, including: If the non-text information is the annotation symbol, identifying the annotation text corresponding to the annotation symbol; After playing the audio information of all the texts in the sentence where the non-text information is located, play the audio information corresponding to the annotation text.

7. The method according to any one of claims 1 to 6, characterized in that: The step of determining the audio information corresponding to the non-text information includes: According to the identification information of the non-text information, the audio information corresponding to the non-text information is determined from a preset audio information library.

8. A text audio playback device, characterized in that: The device comprises: Recognition unit, used to recognize non-text information in the target text; A determination unit, configured to determine audio information corresponding to the non-text information; A playback control unit, configured to play the target text audibly according to the audio information corresponding to the non-text information; Among them, if the non-text information is a characteristic font style of text, then while playing the audio information corresponding to the text with the characteristic font style, the audio information corresponding to the characteristic font style is played as the background sound of the audio information corresponding to the text with the characteristic font style.

9. A terminal device comprising a speaker, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images