Control method of application and electronic device, apparatus and medium thereof

Abstract

The application relates to an application control method, an electronic device, an apparatus and a medium, the application control method comprising: in N executable controls in a first display interface of an application, determining, through a first voice instruction, that M executable controls correspond to the first voice instruction, and displaying second identification information for the M executable controls; and in response to a second voice instruction including the second identification information being received, selecting and running an executable control corresponding to the second identification information included in the second voice instruction from the M executable controls. Through the method and the technical scheme, the electronic device can prompt the executable control of the application containing repeated first identification information in the screen, and the electronic device displays non-repeated second identification information for the control, so that the electronic device can identify the executable control according to the voice issued by the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of mobile terminals, and particularly to a control method of an application, and an electronic device, an apparatus and a medium. BACKGROUND

[0002] In the field of voice control applications of existing electronic devices, a user can control the executable controls of an application display interface through voice control according to the identification information of the executable controls. However, in the case that there are multiple executable controls in the display interface of the application, the identification information of the multiple executable controls can contain repeated contents. When the user's voice contains the repeated contents, the electronic device will not be able to identify the executable controls.

[0003] For example, as shown in Figure 1 (a), a user 100 starts a photo application 300 on an electronic device 200, and a display interface of the photo application 300 displays four executable controls, each of which corresponds to a photo, and the identification information of the four executable controls is “turtle”, “goldfish”, “swan” and “turtle” respectively. Here, the identification information of the photo can be a label marked on the photo when the photo is collected and saved in a storage area of the electronic device 200. Since the label is not checked for uniqueness, there can be multiple photos with repeated identification information. At this time, as shown in Figure 1 (b), when the electronic device 200 collects the “turtle” voice issued by the user 100, the electronic device 200 will not be able to confirm which photo with the identification information of “turtle” the user 100 needs to browse, for example, the electronic device 200 prompts the user 100 that there are multiple objects in the display interface of the electronic device 200, and cannot be determined, thus reducing the user's experience of controlling the photo application 300 through voice. SUMMARY

[0004] The purpose of the present application is to provide a control method of an application, and an electronic device, an apparatus and a medium. Through the method of the present application, through the technical solution of the present application, the electronic device can prompt the executable controls of the application containing repeated first identification information in the screen, at the same time, the electronic device displays the non-repeated second identification information for the controls, so that the electronic device can identify the executable controls according to the voice issued by the user.

[0005] The first aspect of the present application provides a control method of an application for an electronic device, characterized in that it comprises:

[0006] displaying a first display interface of an application, the first display interface comprising N executable controls capable of being controlled by voice, the executable controls displaying first identification information, wherein N is an integer;

[0007] receive a first voice instruction of a user;

[0008] In a case where the M executable controls correspond to the first voice instruction, display second identification information on the plurality of executable controls to prompt that there are the plurality of executable controls, where M is an integer greater than or equal to 2 and less than or equal to N.

[0009] In response to the received second voice instruction including the second identification information, select and execute the executable control corresponding to the second identification information included in the second voice instruction from the M executable controls.

[0010] That is, in the embodiments of the present application, the electronic device can be a smart television, and the application can be a photo application running on the smart television. For example, after the smart television starts the photo application, a photo list interface of the photo application is displayed in the screen of the smart television, and the photo list interface can be the first display interface. In the photo list interface, four photo controls are displayed, and the four here can be N. After the smart television starts the voice recognition function, the four photo controls display respective labels as "turtle", "goldfish", "swan", and "turtle", and the labels here can be the first identification information. After the smart television receives and recognizes the voice "turtle" issued by the user, i.e., the first voice instruction, the smart television determines that the labels of two photo controls contain "turtle", and the two here can be M. The smart television displays "1" and "2" in the upper left corners of the two "turtle" photo controls in sequence, and the "1" and "2" here can be the second identification information. After the smart television receives and recognizes the voice "1" issued by the user, i.e., the second voice instruction, the smart television determines that the user selects the "turtle" photo control with the second identification information "1".

[0011] In a possible implementation of the first aspect, the executable control is configured to enable the application to switch from the first display interface to a second display interface.

[0012] That is, in the embodiments of the present application, for example, the smart television runs the "turtle" photo control with the second identification information "1", and enters the photo detailed interface corresponding to the "turtle" photo control, i.e., the second display interface.

[0013] In a possible implementation of the first aspect, the first identification information of the executable control is displayed based on the electronic device starting the voice recognition function.

[0014] That is, in the embodiments of the present application, for example, after the smart television starts the voice recognition function, the four photo controls in the photo list interface of the photo application display respective labels as "turtle", "goldfish", "swan", and "turtle".

[0015] In a possible implementation of the first aspect, receiving the first voice instruction of the user includes:

[0016] receive the first voice instruction issued by the user, and identify the first text content from the first voice instruction.

[0017] That is, in the embodiments of the present application, for example, the user issues the voice of "turtle", that is, the first voice instruction; the smart TV identifies the text content of "turtle" from the voice of "turtle", that is, the first text content.

[0018] In a possible implementation of the first aspect, in the case that the M executable controls correspond to the first voice instruction, the second identification information is displayed on the plurality of executable controls to prompt the existence of the plurality of executable controls.

[0019] The M executable controls are determined to correspond to the first voice instruction by determining that the first text content is contained in the first identification information of the M executable controls, and the second identification information is displayed on the plurality of executable controls.

[0020] That is, in the embodiments of the present application, for example, the smart TV determines that the labels of the two photo controls in the photo list interface of the photo application contain "turtle", and the smart TV displays "1" and "2" in the upper left corners of the two photo controls of "turtle" in sequence, where "1" and "2" can be the second identification information.

[0021] In a possible implementation of the first aspect, prompting the plurality of executable controls includes: setting the plurality of executable controls to a first color, and setting elements other than the plurality of executable controls to a second color.

[0022] That is, in the embodiments of the present application, for example, the smart TV can set the two photo controls containing "turtle" to red, that is, the first color; and set the two photo controls of "goldfish" and "swan" to gray, that is, the second color.

[0023] In a possible implementation of the first aspect, the second identification information has uniqueness.

[0024] That is, in the embodiments of the present application, the second identification information can be natural numbers arranged in sequence, for example, "1", "2", "3", "4".

[0025] In a possible implementation of the first aspect, the executable control simultaneously displays the second identification information and the first identification information.

[0026] That is, in the embodiments of the present application, for example, in addition to displaying "turtle" on the two photo controls of "turtle", "1" and "2" are also displayed simultaneously.

[0027] In a possible implementation of the first aspect, the second identification information replaces the display of the first identification information.

[0028] That is, in the embodiments of the present application, for example, "turtle" is not displayed on the two photo controls of "turtle", but "1" and "2" are displayed.

[0029] In a possible implementation of the first aspect, the application includes one of an application of an application software type and an application of an operating system type, and the application contains at least one display interface.

[0030] That is, in the embodiments of the present application, the application of the application software type can be an application for implementing a specific function of the electronic device, for example, the application of the application software type can be a photo application, a video playing application. The application of the operating system type can be an application for implementing management and setting of the electronic device, for example, a menu application of the operating system of the electronic device, a system setting application.

[0031] The second aspect of the present application provides a device for developing a runnable control of an application, characterized in that, comprising:

[0032] A control design module, the control design module is used for coding the runnable control, and after the coding of the runnable control is completed, the style of the runnable control is previewed;

[0033] A code analysis module, the code analysis module is used for checking whether the code of the runnable control contains repeated contents for voice control;

[0034] A code prompt module, the code prompt module is used for prompting the runnable control containing the repeated contents for voice control detected by the code analysis module.

[0035] In a possible implementation of the first aspect, the code prompt module sets the repeated contents for voice control as highlighted display.

[0036] That is, in the embodiments of the present application, for example, the code prompt module can set the color of the repeated contents for voice control in the runnable control to be different from other runnable controls, for example, to be red.

[0037] The third aspect of the present application provides an electronic device, characterized in that, comprising:

[0038] A memory, the memory stores instructions;

[0039] A processor, the processor and the memory are coupled, when the program instructions stored in the memory are executed by the processor, the electronic device executes the control method of the application provided in the first aspect.

[0040] The fourth aspect of the present application provides a readable medium, and the readable medium stores instructions, and the instructions are characterized in that when the instructions run on the readable medium, the readable medium executes the control method of the application provided in the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 (a) and Figure 1 (b) shows a scenario of controlling an application by voice recognition according to an embodiment of the present application;

[0042] Figure 2 (a) to Figure 2 (f) shows another scenario of controlling an application by voice recognition according to an embodiment of the present application;

[0043] Figure 3 A hardware structure block diagram of an electronic device according to an embodiment of the present application is shown;

[0044] Figure 4 A method flow chart of controlling an application by voice recognition according to an embodiment of the present application is shown;

[0045] Figure 5 (a) and Figure 5 (b) shows a code structure diagram of first identification information and second identification information in a runnable control according to an embodiment of the present application;

[0046] Figure 6 A method flow chart of another scenario of controlling an application by voice recognition according to an embodiment of the present application is shown;

[0047] Figure 7 (a) to Figure 7 (c) shows another scenario of controlling an application by voice recognition according to an embodiment of the present application;

[0048] Figure 8 A module structure block diagram of a development device for developing a runnable control of an application according to an embodiment of the present application is shown;

[0049] Figure 9 A method flow chart of developing a runnable control of an application according to an embodiment of the present application is shown;

[0050] Figure 10 A scenario diagram of a development device prompting a runnable control containing repeated content according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] Embodiments of the present application include but are not limited to an application control method and its electronic device, device and medium.

[0052] To solve the problem that the electronic device cannot identify the executable control of the application containing repeated content, the application provides an application control method. Through the technical solution of the application, the electronic device can prompt the executable control of the application containing the repeated first identification information in the screen, and the electronic device displays the second identification information that is not repeated for the control, so that the electronic device can identify the executable control according to the voice issued by the user. The executable control herein refers to a control that can be triggered by gesture operation or voice recognition in the display interface of the application to change the display content in the display interface of the application.

[0053] To make the objects, technical solutions and advantages of the application clearer, the embodiments of the application will be further described in detail below with reference to the drawings.

[0054] Figure 2 (a) to Figure 2 (f) shows the scenario of application control in the embodiment of the application, as Figure 2 As shown in (a) and (b), the user 100 starts the photo application 300 on the electronic device 200, and the screen of the electronic device 200 displays the current display interface of the photo application 300. After the user 100 starts the voice control function of the electronic device 200, the first identification information is displayed on the executable control of the current display interface of the photo application 300. The first identification information is used to represent an attribute of the executable control, for example, in the photo application 300, the first identification information can be the label of the photo. Figure 2 In (c), the executable control can be a photo control, and the first identification information can be the label of the photo. The user 100 issues a voice instruction to the electronic device 200, and the electronic device 200 identifies the text content in the voice and searches for the executable control containing the text content in the first identification information in the current display interface of the photo application 300. In the case that the electronic device 200 finds multiple executable controls containing the aforementioned text content in the first identification information in the current display interface, the electronic device 200 prompts the user with the multiple executable controls in the current display interface of the photo application 300, and adds the second identification information to the multiple executable controls. For example, as shown in Figure 2 As shown in (d), the electronic device 200 can dim the current display interface of the photo application 300 and highlight the multiple executable controls to prompt the user. At the same time, the electronic device 200 adds the serial number of the executable control as the second identification information to the multiple executable controls. Then, the user 100 issues a voice to the electronic device 200 again according to the second identification information of the multiple executable controls, the electronic device 200 identifies the text content in the voice and searches for the executable control containing the text content in the second identification information among the multiple executable controls prompted to the user 100. In the case that a single executable control is found, the electronic device 200 triggers the executable control.

[0055] In the application control method described above, the electronic device 200 can be various terminal devices, such as, but not limited to, a smart television, a laptop computer, a desktop computer, a tablet computer, a mobile phone, a server, a wearable device, a head-mounted display, a mobile email device, a portable game console, a portable music player, a reader device, or other terminal devices capable of accessing a network. In some embodiments, the embodiments of the present application can also be applied to a wearable device worn by a user. For example, a smart watch, a bracelet, jewelry (e.g., a device made into a decorative item such as an earring, a bracelet, etc.), or glasses, etc., or as part of a watch, a bracelet, jewelry, or glasses, etc. For simplicity of illustration, the following will take the smart television 200 as an example of the electronic device 200 to introduce the solution of the present application.

[0056] Figure 3 According to an embodiment of the present application, a structural schematic diagram of the smart television 200 is shown. Specifically, as shown in Figure 2 The smart television 200 includes a processor 201, a memory 202, a microphone 203, a voice recognition module 204, an interface management module 205, a power switch 206, a communication processing module 207, a display screen 208, a bus 209, etc.

[0057] The processor 201 can be used to read and execute computer readable instructions. In a specific implementation, the processor 201 can mainly include a controller, an arithmetic unit, and a register. Among them, the controller is mainly responsible for instruction decoding and sending control signals for the operation of the instruction. The arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logic operations, etc., and can also perform address operations and conversion. The register is mainly responsible for saving the register operands and intermediate operation results temporarily stored in the process of instruction execution, etc. In a specific implementation, the hardware architecture of the processor 201 can be an application-specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.

[0058] The memory 202 is coupled with the processor 201, and is used to store various software programs and / or groups of instructions. In a specific implementation, the memory 202 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more disk storage devices, flash devices, or other non-volatile solid-state storage devices. The memory 202 can store an operating system, such as an embedded operating system such as uCOS, VxWorks, RTLinux, etc. The memory 202 can also store a communication program, which can be used to communicate with a mobile phone, one or more servers, or additional devices.

[0059] The microphone 203 is used to collect the voice issued by the user.

[0060] The speech recognition module 204 is configured to acquire the speech uttered by the user and recognize the text content from the speech. The speech recognition module 204 can generate feature vectors respectively describing the speech uttered by the user through a speech feature extraction algorithm, and arrange the feature vectors in time sequence to obtain the feature vectors. The feature of the speech can be Linear Predictive Coding (LPC) feature, Linear Predictive Cepstral Coefficients (LPCC) feature, Mel-scale Frequency Cepstral Coefficients (MFCC) feature or Linear Prediction Based Mel Frequency Cepstral Coefficient (LBPMFCC) feature of the speech uttered by the user.

[0061] The interface management module 205 is configured to manage the display interface of the application running on the smart television 200. For example, the interface management module 205 can construct the display interface of the application, add controls such as a control for displaying text, a control for displaying pictures, etc. on the display interface.

[0062] The power module 206 can include a power supply, a power management component, etc. The power supply can be a battery. The power management component is configured to manage charging of the power supply and power supply to other modules.

[0063] The communication processing module 207 can include a wireless communication processing module (not shown) and a wired local area network (LAN) communication processing module (not shown).

[0064] The wireless communication processing module can further include a cellular mobile communication processing module (not shown). The cellular mobile communication processing module can communicate with other devices (such as a server) through cellular mobile communication technology.

[0065] The wired LAN communication processing module can be used for communication with other devices in the same LAN through the wired LAN, and can also be used for connecting to a wide area network (WAN) through the wired LAN, and can communicate with devices in the WAN.

[0066] The display screen 208 can be used to display images, videos, etc. The display screen 208 can be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display screen, an active-matrix organic light emitting diode (AMOLED) display screen, a flexible light-emitting diode (FLED) display screen, a quantum dot light emitting diodes (QLED) display screen, etc.

[0067] The bus 209 is a common communication trunk for transmitting information between the above-mentioned functional components and modules in the smart television 200.

[0068] It can be understood that, Figure 3 The illustrated structure does not constitute a specific limitation on the smart television 200. In some other embodiments of the present application, the smart television 200 can include more or fewer components than the illustration, or combine some components, or split some components, or different component arrangement. The illustrated components can be implemented in hardware, software or a combination of software and hardware.

[0069] The application control method of the present application will be described in detail below with reference to Figures 2 to 5 For example, taking the case that the user controls a runnable control in the current display interface of the photo application 300 running on the smart television 200 by voice as an example, the application control method of the present application will be described in detail. Specifically, as shown in Figure 4 The application control method provided by the present application includes the following steps:

[0070] S401: Start the photo application 300.

[0071] For example, as shown in Figure 2 (a), after the user 100 turns on the smart television 200, the user 100 can open the photo application 300 by selecting and clicking the icon of "photo" in the user interface (UI) of the smart television 200 through the remote controller of the smart television 200. After the smart television 200 responds to the instruction, the photo application 300 is opened, and the current display interface of the photo application 300 is displayed on the screen of the smart television 200. As shown in Figure 2(b) shows that the current display interface of the photo application 300 includes the photo list interface 3001 of the photo application 300. The photo list interface 3001 contains four executable controls, and for the executable controls in the photo list interface 3001, the type of the executable control can be a photo control, which can refer to each photo in the photo list interface 3001. The user can trigger the photo control by a gesture or by voice, and after the photo control is triggered, the smart TV 200 can enter a photo preview interface from the photo list interface 3001.

[0072] S402: Start the voice control function.

[0073] For example, the smart TV 200 can receive an instruction sent by the user 100 pressing a button of the voice control function set on the remote controller of the smart TV 200 to start the voice control function of the smart TV 200. At this time, as shown in Figure 2 (c) shows that the interface management module 205 of the smart TV 200 displays the first identification information in the photo controls in the photo list interface 3001 of the photo application 300. The first identification information of the photo control can be the label of the photo. For example, the photo list interface 3001 includes four photo controls, and the first identification information of each of the four photo controls is "turtle", "goldfish", "swan", and "turtle". The first identification information here can be the attribute of each photo control, for example, the first identification information can be the label of the photo. Figure 5 (a) shows the code format of the above-mentioned four photo controls "turtle", "goldfish", "swan", and "turtle". The first identification information of the four photo controls can be the name of the photo control, that is Figure 5 The "app:name" attribute in the code of the photo control described in (a) corresponds to the value. After the smart TV 200 starts the voice control function, the interface management module 205 displays the corresponding first identification information in the above-mentioned four photo controls.

[0074] In another embodiment of the present application, the smart TV 200 can also enter the voice recognition mode by interacting with the user 100. For example, after the smart TV 200 receives the user saying "start the voice control function" to the smart TV 200, the smart TV 200 starts the voice control function.

[0075] S403: Acquire the voice sent by the user and identify the text content corresponding to the voice.

[0076] In the state that the voice control function is enabled, the voice recognition module 204 of the smart television 200 listens to the voice issued by the user to the smart television 200 through the microphone 203, for example, the user issues the voice of "turtle" to the smart television 200, the smart television 200 can collect the voice issued by the user through the microphone 203 and save it in the storage 202 of the smart television, and then recognize the text content corresponding to the voice through the voice recognition module 204 of the smart television 200. In some other embodiments of the present application, in addition to the voice of "turtle", the user can also issue the voice of "goldfish" to the smart television 200.

[0077] It can be understood that in other embodiments, the smart television 200 can also collect the voice issued by the user through an audio collection device including a microphone, which can be a part of the smart television 200 or an independent device. When the audio collection device is an independent device outside the smart television 200, the collected audio signal can be sent to the smart television 200 through a communication connection with the smart television 200, for example, the audio collection device can be a sound pickup device, a recording microphone, etc.

[0078] The voice recognition module 204 of the smart television 200 can input the saved voice into a voice neural network model to calculate the text content corresponding to the voice. For example, the voice neural network model herein can be a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a long short-term memory (LSTM), etc. In the case that the voice neural network model is a convolutional neural network, the neural network can include multiple convolutional layers, pooling layers and output layers, and the input data of the neural network is the voice issued by the user collected by the smart television 200. For example, after the smart television 200 inputs the voice of "turtle" issued by the user into the neural network, the corresponding text content "turtle" is obtained.

[0079] S404: confirming whether there is only one executable control in the current display interface of the photo application 300, the first identification information of the executable control containing the text content.

[0080] After obtaining the text content corresponding to the user's voice, the interface management module 205 of the smart TV 200 can traverse the runnable controls within the current display interface of the photo application 300, obtain the first identification information corresponding to each runnable control, and determine whether there are multiple runnable controls whose first identification information contains the text content corresponding to the user's voice. If there is a runnable control containing the text content in the current display interface of the photo application 300, S406 is executed, and the smart TV 200 triggers the runnable control; if there are multiple runnable controls, S405 is executed, and the smart TV 200 prompts the user about the multiple runnable controls in the current display interface of the photo application 300, and displays the second identification information for the multiple runnable controls; if there are no runnable controls, the smart TV 200 can prompt the user that no runnable control can be found. For example, the smart TV 200 can prompt "No runnable control found, please resend the voice" in the current display interface of the photo application 300.

[0081] by Figure 2 (c) Taking the photo overview interface 3001 of the photo application 300 as an example, for the four photo controls of the photo overview interface 3001, the first identification information of each is "turtle", "goldfish", "swan" and "turtle". The interface management module 205 of the smart TV 200 traverses the first identification information of the above four photo controls and determines that the first identification information of two photo controls contains the text content "turtle" corresponding to the voice of the user.

[0082] In some other embodiments of this application, after the interface management module 205 of the smart TV 200 traverses the first identification information of the above four operable controls and determines that only one operable control has a first identification information containing the text content "goldfish" corresponding to the voice issued by the user, the smart TV 200 executes S406.

[0083] S405: In the current display interface of the photo application 300, prompt the user with a plurality of operable controls and display second identification information for the operable controls.

[0084] In the case that the interface management module 205 of the smart TV 200 determines that the first identification information of the plurality of executable controls existing in the current display interface of the photo application 300 contains the text content corresponding to the voice issued by the user, the interface management module 205 of the smart TV 200 prompts the plurality of executable controls to the user in a highlighted manner in the current display interface of the photo application 300, and displays the second identification information of the plurality of executable controls. Then, the smart TV 200 returns to step S403, and the user issues a voice to the smart TV 200 again according to the second identification information, so that the smart TV 200 can acquire the voice issued by the user again, recognize the text content corresponding to the voice, select one executable control from the plurality of executable controls, and trigger the executable control.

[0085] For example, Figure 2 In the embodiment shown in (d), after the interface management module 205 of the smart TV 200 determines that the first identification information of the two photo controls existing in the photo list interface 3001 of the photo application 300 contains the text content "turtle" corresponding to the voice issued by the user, the interface management module 205 thickens the borders of the two photo controls, and darkens the brightness of the other controls except the two photo controls containing "turtle", so that the two executable controls are highlighted in the photo list interface 3001 of the photo application 300. At the same time, the interface management module 205 displays the second identification information on the two photo controls. Figure 5 (b) shows the code format of the above two "turtle" photo controls, and the second identification information thereof can be Figure 5 The "voice_event" attribute in the code of the photo control described in (b) corresponds to a value. As can be seen, in step S402, in the case that the interface management module 205 uses the first identification information to identify the photo control, the value corresponding to the "voice_event" attribute in the code of the photo control is empty. In the case that the interface management module 205 determines that the plurality of photo controls exist in the photo list interface 3001 of the photo application 300, the interface management module 205 can number the plurality of photo controls in sequence, and set the value corresponding to the number as the value corresponding to the "voice_event" attribute, that is, the second identification information. In this way, the interface management module 205 can prompt the user 100 to perform voice control by displaying the second identification information on the photo control. In other embodiments of the present application, for example, the interface management module 205 can also set the above two "turtle" photo controls to red and the other controls to gray, for prompting the user that the two "turtle" photo controls.

[0086] For example, the interface management module 205 labels the two "turtle" photo controls as "1" and "2" according to their positions in the photo overview interface 3001, from left to right and from top to bottom. Then, the "voice_event" attribute of these two photo controls is set to "1" and "2" respectively. Simultaneously, in the photo overview interface 3001, "1" and "2" are displayed sequentially in the upper left corner of the two "turtle" photo controls, prompting the user 100 to select one of the two executable controls to trigger by uttering the voice commands "1" or "2".

[0087] S406: Triggers a runnable control.

[0088] For example, in Figure 2 In the embodiment shown in (e), after user 100 pronounces the voice "1" again and it is recognized by smart TV 200, smart TV 200 determines that the user has selected the "turtle" photo control with the second identifier information "1". Then, smart TV 200 enters the following... Figure 2 (f) shows the photo details interface 3002, which displays a photo of a "turtle".

[0089] In some other embodiments of this application, if the user's voice input is "goldfish," the interface management module 205 of the smart TV 200 can uniquely identify the "goldfish" photo control using the first identification information. Then, the smart TV 200 enters the photo details interface 3002, displaying the "goldfish" photo.

[0090] After the introduction Figures 2 to 5 Following the technical solution described above, where user 100 controls the photo application 300 of smart TV 200 via voice, the following section continues using smart TV 200 and video playback application 400 as examples, combined with... Figure 6 This application will now describe another application control method in detail.

[0091] Figure 6 The proposed solutions and Figures 2 to 5 The difference in the described scheme is that the first identification information of the runnable control of the video playback application 400 can be an attribute of the runnable control, such as the name of the runnable control, and the first identification information of the runnable control is already displayed in the display interface 401 of the video playback application 400 before the user starts the voice control function of the smart TV 200.

[0092] Specifically, Figure 6 The application control methods involved include the following steps:

[0093] S601: Start video playback application 400.

[0094] S601: Opening the video playing application.

[0095] S601: Opening the video playing application. Figure 7 As shown in FIG. 4(a), the current display interface 401 of the video playing application 400 includes a first video column 4001, a second video column 4002 and a title column 4003. Each of the first video column 4001, the second video column 4002 and the title column 4003 contains a plurality of executable controls with first identification information. For example, the first video column 4001 includes four executable controls with respective first identification information of "current news", "breaking news", "today's review" and "king of comedy". The second video column 4002 includes five executable controls with respective first identification information of "recommended movie", "Fuyao", "Jingangchuan", "Justice League" and "Youth Has You". The title column 4003 includes six executable controls, such as "recommendation", "movie", "TV series", "variety show", "children" and "mine".

[0096] The executable controls in the first video column 4001 and the second video column 4002 can be video controls, which, when triggered, can lead the video playing application 400 to a video playing interface. The first identification information of the video controls can be video names, video introductions, etc. The executable controls in the title column 4003 can be title selection controls, which, when triggered, can lead the video playing application 400 to a display interface of a title corresponding to the title selection control. The first identification information of the title selection controls can be title names.

[0097] S602: Starting the voice control function.

[0098] S602: Starting the voice control function.

[0099] S603: Acquiring the voice sent by the user and identifying the text content corresponding to the voice.

[0100] S603 here is similar to S403 mentioned above. When the voice control function is enabled, the smart TV 200 listens to the voices spoken by the user. For example, if the user says "news" to the smart TV 200, the smart TV 200 can collect the user's voice through the microphone 203 and store it in the smart TV's memory 202.

[0101] It is understandable that the Smart TV 200 can obtain the corresponding text content "news" after the user's "news" voice is transmitted through the speech recognition neural network on the Smart TV 200.

[0102] S604: Confirm whether there is a runnable control in the current display interface 401 of the video playback application 400, and the first identification information of the runnable control contains the text content.

[0103] S604 here is similar to S404 described above. After obtaining the text content corresponding to the user's voice, the smart TV 200 iterates through the runnable controls in the current display interface of the video playback application 400, obtains the first identifier information corresponding to each runnable control, and determines whether there are multiple runnable controls whose first identifier information contains the text content corresponding to the user's voice. If there is a runnable control containing the text content in the current display interface of the video playback application 400, S606 is executed, and the smart TV 200 triggers the runnable control; otherwise, S605 is executed. If there are multiple runnable controls, the smart TV 200 prompts the user with the multiple runnable controls in the current display interface of the video playback application 400; if there are no runnable controls, the user is prompted that no runnable controls can be found.

[0104] For example, such as Figure 7 As shown in (a), for the four executable controls of the first video bar 4001, their respective first identifier information is "Current News", "Breaking News", "Today's Commentary", and "King of Comedy". The second video bar 4002 includes five executable controls. The smart TV 200 iterates through the first identifier information of the above four executable controls and determines that the first identifier information of the "Current News" and "Breaking News" executable controls contains the text content "News" corresponding to the user's voice. Then, for the second video bar 4002 and the title bar 4003, the smart TV 200 can also perform the same operation as it performed on the first video bar 4001 to determine whether the second video bar 4002 and the title bar 4003 include executable controls that contain the text content "News" corresponding to the user's voice.

[0105] S605: In the current display interface of the video playing application 400, the user is prompted with a plurality of executable controls, and the second identification information is displayed for the executable controls.

[0106] S605 here is similar to S405 described above. In the current display interface of the video playing application 400, after the smart television 200 determines that the first identification information of the plurality of executable controls contains the text content corresponding to the voice issued by the user, the smart television 200 prompts the user with the plurality of executable controls in a highlighted manner in the current display interface of the video playing application 400, and displays the second identification information for the plurality of executable controls, and continues to S603 to prompt the user to issue the voice again according to the second identification information, so that the smart television 200 can acquire the voice issued by the user again, and recognize the text content corresponding to the voice, so that one of the plurality of executable controls is selected to be triggered.

[0107] For example, as shown in Figure 7 (b), after the smart television 200 determines that the first identification information of the two executable controls of "current news" and "breaking news" contains the text content "news" corresponding to the voice issued by the user, the smart television 200 thickens the borders of the two executable controls of "current news" and "breaking news", and darkens the brightness of other controls except the two executable controls of "current news" and "breaking news", so that the two executable controls of "current news" and "breaking news" are highlighted in the current display interface of the video playing application 400. At the same time, the smart television 200 also numbers the two executable controls of "current news" and "breaking news", and identifies "current news" as "1" and "breaking news" as "2", that is, the second identification information "1" and "2" is displayed on the two executable controls of "current news" and "breaking news", and the second identification information "1" and "2" is displayed in the upper left corner of the two executable controls of "current news" and "breaking news", to prompt the user 100 to trigger one of the two executable controls of "current news" and "breaking news" by issuing the voice of "1" or "2".

[0108] S606: Triggering the executable control.

[0109] S606 here is similar to S406 described above. As shown in Figure 7 (c), after the user issues the voice of "1" and it is recognized by the smart television 200, the smart television 200 determines that the user selects the executable control of "current news", and triggers the executable control of "current news", and displays the display interface 402 of the executable control of "current news" in the video playing application 400.

[0110] In addition to the photo application 300 and the video playing application 400 described above, the application control method in the embodiments of the present application can also be applied to other types of applications running on the smart TV 200 with a display interface, such as a menu application of the operating system of the smart TV 200, a system setting application.

[0111] In addition to the application control method described above, the embodiments of the present application also disclose a development device of a runnable control of an application. The development device can be applied to develop the runnable control in the photo application 300 as shown in Figure 2 and the video playing application 400 as shown in Figure 7 The runnable control of an application herein is developed by a developer using the development device 500 running on a development device and saved in a storage area of the development device. The development device can include but is not limited to a laptop, a desktop computer, a tablet computer, a server and other computer devices capable of being used for software development. As shown in Figure 8 the development device 500 can include:

[0112] A control design module 501: the control design module is used by the developer to code the runnable control. For example, in the case that the runnable control belongs to a visible control in the display interface of the application, the developer can preview the runnable control through the control design module after completing the coding of the runnable control. Meanwhile, the control design module can also pre-store a plurality of templates of the runnable control in the memory of the development device, and the developer can directly select a template from the templates of the runnable control and code based on the template.

[0113] A code analysis module 502: the code analysis module is used to check the code of the runnable control, and check whether the code of the runnable control contains repeated contents for voice control. If yes, the code analysis module prompts the developer through the code prompt module. The contents for voice control herein can be pre-configured by the developer in the memory of the development device. For example, for the runnable control of the photo application as described in Figure 5 the contents for voice control of the runnable control are the value corresponding to the "app:name" attribute, and the developer can pre-configure the "app:name" attribute in the memory of the development device, so that the code analysis module can check whether the "app:name" attribute contains repeated contents when checking the runnable control of the photo application.

[0114] The code prompt module 503 is configured to prompt the developer with the code of the executable control detected by the code analysis module. For example, when the code of the executable control has an error, the code prompt module can add an underline under the code with the error, and highlight the code and color the code red to prompt the developer. Meanwhile, the code prompt module can also display an error prompt on the console of the development device.

[0115] The process of the developer developing the executable controls of the photo application 300 using the development device of the present application will be described below, as shown in Figure 9 , including:

[0116] S901: Receive the code of the executable control of the display interface of the application developed by the developer and save it.

[0117] The developer uses the control design module to complete the coding of the four photo controls of the photo application 300, and saves them in the storage area of the development device.

[0118] S902: Verify the code of the executable control.

[0119] The code analysis module performs repetitive verification on the code of the photo controls saved in the development device. For example, continuing to refer to Figure 9 For example, the code analysis module can obtain the respective "app:name" attributes from the code of the four executable controls according to the "app:name" attribute set by the developer for voice control, and verify whether the values corresponding to the "app:name" attributes of each executable control contain duplicate content.

[0120] S903: Confirm whether there are two or more executable controls in the code that contain the same content for voice control.

[0121] In the case where the code analysis module determines that there are two or more executable controls in the code that contain the same content for voice control, S904 is executed, and the code analysis module can send the executable controls with the same content for voice control to the code prompt module, and the code prompt module prompts the developer of the executable controls with the same content in the development device. If not, S905 is executed, and the code analysis module determines that the code of the executable control is normal.

[0122] S904: Prompt the developer of the executable controls containing the same content for voice control.

[0123] The code hinting module can add an underline under the code of the runnable control that contains the duplicated content, and highlight the code with red color to hint, and at the same time, the code hinting module can also hint the runnable control that contains the same content for voice control in the prompt information area of the development device. For example, as shown in Figure 10 The code analysis module determines whether the values corresponding to the "app:name" attributes of two of the four runnable controls contain the duplicated content "turtle". The code analysis module notifies the code hinting module that the "app:name" attributes of the two runnable controls contain the duplicated content "turtle". The code hinting module can hint "the "app:name" attributes in lines 4, 22 of the code contain the same content" in the prompt information area of the development device. In this case, the development device receives the coding of the runnable controls with the same content by the developer again, returning to S701.

[0124] S905: Complete the compilation of the code of the runnable control.

[0125] The code analysis module confirms that the code of the runnable control is correct, and the development device can compile the code of the runnable control and execute.

[0126] It should be understood that, although the terms "first", "second", etc. can be used herein to describe various features, these features should not be limited by these terms. These terms are only used to distinguish one feature from another, and cannot be understood as indicating or implying relative importance. For example, without departing from the scope of the example embodiments, a first feature can be called a second feature, and similarly a second feature can be called a first feature.

[0127] In addition, various operations will be described as multiple discrete operations, in a manner that is most helpful in understanding the illustrative embodiments; however, the order of description should not be construed as to imply that these operations are necessarily order dependent. Many of the operations described can be handled in parallel, concurrently or in some cases simultaneously. Further, the described operations can be rearranged or reordered in other ways. The disclosure is not limited as to the order of the operations described. The processes depicted in the exemplary figures can comprise additional operations not included in the figures. These processes can also comprise fewer operations than those depicted in the exemplary figures. The processes depicted in the exemplary figures can be implemented using one or more devices selected from a group comprising a processor, a controller, a microcontroller, a microprocessor, a programmable logic device, a computer, a system on chip, and the like.

[0128] Reference in the specification to "one embodiment", "an embodiment", "the illustrative embodiment", etc. means that a described embodiment can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of those in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0129] The terms "comprise", "have" and "include" are synonymous, unless the context requires otherwise. The phrase "A / B" means "A or B". The phrase "A and / or B" means "(A), (B), or (A and B)".

[0130] As used herein, the term "module" can refer to, be part of, or include a memory (shared, dedicated, or group) that stores one or more software or firmware programs, an application-specific integrated circuit (ASIC), an electronic circuit, and / or a processor (shared, dedicated, or group) that executes the software or firmware programs to provide the described functionality, combinational logic circuit, and / or other suitable components that provide the described functionality.

[0131] In the drawings, some of the structural or methodological acts can be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order is not necessarily required. Instead, in some embodiments, the acts can be performed in an order different than those described in the illustrative figures. Further, some of the acts can not be required in some embodiments, or can be combined with other acts in some embodiments, or can be performed in conjunction with other acts in some embodiments.

[0132] The embodiments of the present application are described in detail above with reference to the accompanying drawings, but the use of the technical solutions of the present application is not limited to the various applications, structures and variations mentioned in the patent embodiments. Various structures and variations can be easily implemented with reference to the technical solutions of the present application to achieve the various beneficial effects mentioned herein. Various changes made within the knowledge of those skilled in the art without departing from the purpose of the present application shall be attributed to the scope of the present application.

Claims

1. A control method of an application for an electronic device, the method comprising: The application comprises: displaying a first display interface of the application, the first display interface comprising N runable controls capable of being controlled by voice, the runable controls displaying first identification information, wherein N is an integer; receiving a first voice instruction of a user; in a case where M runable controls correspond to the first voice instruction, sequentially numbering the M runable controls according to positions of the M runable controls in the first display interface, setting a numbered value as second identification information of the M runable controls, the second identification information being a value corresponding to a first attribute in codes of the M runable controls, and displaying the second identification information of the M runable controls to prompt that there are multiple runable controls, wherein in a case where the M runable controls are identified based on the first identification information, the value corresponding to the first attribute in the codes of the M runable controls is empty, M is an integer greater than or equal to 2 and less than or equal to N, the runable controls simultaneously display the second identification information and the first identification information or the second identification information replaces the first identification information, the second identification information is unique, and the M runable controls are presented in a highlighted manner; in response to a received second voice instruction comprising the second identification information, selecting and running the runable control corresponding to the second identification information comprised in the second voice instruction from the M runable controls.

2. The method of claim 1, wherein, The runable control is used to make the application switch from the first display interface to a second display interface.

3. The method of claim 1, wherein, The first identification information of the runable control is displayed based on the electronic device starting a voice recognition function.

4. The method of claim 1, wherein, Receiving the first voice instruction of the user comprises: receiving the first voice instruction issued by the user and identifying first text content from the first voice instruction.

5. The method of claim 4, wherein, In a case where M runable controls correspond to the first voice instruction, displaying second identification information of the multiple runable controls to prompt that there are multiple runable controls; determining that the M runable controls correspond to the first voice instruction by determining that the first text content is contained in the first identification information of the M runable controls, and displaying second identification information of the multiple runable controls.

6. The method of claim 1, wherein, Prompting the multiple runable controls comprises: setting the multiple runable controls to a first color and setting elements other than the multiple runable controls to a second color.

7. The method of claim 1, wherein, The application comprises one of an application software type application and an operating system type application, and the application comprises at least one display interface.

8. An electronic device, comprising: The application comprises: a memory storing instructions; a processor coupled with the memory, when program instructions stored in the memory are executed by the processor, the electronic device executes the control method of the application in any one of claims 1 to 7.

9. A readable medium having stored therein instructions, characterized in that, When the instructions run on the readable medium, the readable medium executes the control method of the application in any one of claims 1 to 7.

Images