Remote control method and device, terminal and storage medium

Abstract

The embodiment of the invention discloses a remote control method and device, a terminal and a storage medium, and belongs to the technical field of remote control. The method is applied to a first terminal, and comprises the following steps: sharing a screen picture with a second terminal in a process of carrying out an audio and video call with the second terminal; under the condition that the voice data transmitted by the second terminal are received, identifying the voice data, and generating an operation execution instruction; and based on the operation execution instruction, executing an operation indicated by the operation execution instruction in a current interface. By adopting the scheme provided by the embodiment of the invention, the remote control process can be simplified, and the remote control efficiency can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of remote control technology, and in particular to a remote control method, device, terminal, and storage medium. Background Art

[0002] With the development of Internet technology, more and more communication and interaction between people can be completed online. For example, a control terminal and a controlled terminal can establish remote control, so that the control terminal can remotely assist the controlled terminal.

[0003] In the related art, in the process of establishing remote control, the controlled end needs to perform corresponding triggering operations to generate a controlled request and send it to the controlling end. Only after the controlling end receives the controlled request can the two parties establish remote control, which makes the remote control operation cumbersome. Summary of the Invention

[0004] The present application provides a remote control method, device, terminal, and storage medium. The technical solution is as follows:

[0005] In one aspect, an embodiment of the present application provides a remote control method, which is used for a first terminal and includes:

[0006] During an audio or video call with a second terminal, sharing the screen with the second terminal;

[0007] When receiving voice data transmitted by the second terminal, recognizing the voice data and generating an operation execution instruction;

[0008] Based on the operation execution instruction, the operation indicated by the operation execution instruction is executed in the current interface.

[0009] On the other hand, an embodiment of the present application provides a remote control method, which is used for a second terminal, and the method includes:

[0010] During an audio or video call with a first terminal, displaying a screen corresponding to the first terminal;

[0011] transmitting the collected voice data to the first terminal so that the first terminal recognizes the voice data and generates an operation execution instruction;

[0012] After the first terminal executes the operation indicated by the operation execution instruction, the screen image corresponding to the first terminal is updated and displayed.

[0013] On the other hand, an embodiment of the present application provides a remote control device, comprising:

[0014] A screen sharing module, configured to share a screen image with a second terminal during an audio or video call with the second terminal;

[0015] an instruction generating module, configured to, upon receiving voice data transmitted by the second terminal, identify the voice data and generate an operation execution instruction;

[0016] The operation execution module is used to execute the operation indicated by the operation execution instruction in the current interface based on the operation execution instruction.

[0017] On the other hand, an embodiment of the present application provides a remote control device, comprising:

[0018] A screen display module, configured to display a screen corresponding to the first terminal during an audio or video call with the first terminal;

[0019] a data transmission module, configured to transmit the collected voice data to the first terminal, so that the first terminal recognizes the voice data and generates an operation execution instruction;

[0020] The screen updating module is configured to update and display a screen corresponding to the first terminal after the first terminal executes the operation indicated by the operation execution instruction.

[0021] On the other hand, an embodiment of the present application provides a terminal, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the remote control method as described in the above aspects.

[0022] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the remote control method as described in the above aspects.

[0023] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the remote control method described in the above aspects.

[0024] In an embodiment of the present application, during an audio or video call between a first terminal and a second terminal, the first terminal shares its screen with the second terminal, and upon receiving voice data transmitted by the second terminal, the first terminal recognizes the voice data to generate an operation execution instruction, thereby enabling the operation indicated by the operation execution instruction to be executed on the current interface of the first terminal based on the voice signal on the second terminal side, i.e., remote control of the first terminal by the second terminal is achieved. Compared with remote control based on screen-triggered operations, the remote control method proposed in this application can simplify the remote control process and improve the efficiency of remote control. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present application is shown;

[0026] Figure 2 A flowchart of a remote control method provided by an exemplary embodiment of the present application is shown;

[0027] Figure 3 A schematic diagram of a remote control provided by an exemplary embodiment of the present application is shown;

[0028] Figure 4 A flowchart of a remote control method provided by another exemplary embodiment of the present application is shown;

[0029] Figure 5 A schematic diagram of interface switching of a first terminal provided by an exemplary embodiment of the present application is shown;

[0030] Figure 6 A flowchart of a remote control method provided by another exemplary embodiment of the present application is shown;

[0031] Figure 7 A schematic diagram of remote control interaction provided by an exemplary embodiment of the present application is shown;

[0032] Figure 8 shows an interactive schematic diagram of remote control provided by another exemplary embodiment of the present application;

[0033] Figure 9 A structural block diagram of a remote control device provided by an exemplary embodiment of the present application is shown;

[0034] Figure 10 shows a structural block diagram of a remote control device provided by another exemplary embodiment of the present application;

[0035] Figure 11 A schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0037] In this document, "plurality" refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0038] Remote control refers to the process by which a control terminal can remotely operate and manage a controlled terminal. By establishing remote control between the control terminal and the controlled terminal, the control terminal can provide remote assistance to the controlled terminal. For example, remote control can guide elderly people to use devices such as smartphones and tablets to perform various complex terminal operations such as adding contacts, setting alarms, and configuring system settings.

[0039] In the related art, establishing remote control requires both the controlling and controlled terminals to trigger operations to enable screen sharing. The controlled terminal then triggers operations to generate a control request and sends it to the controlling terminal. Only after the controlling terminal receives the control request can remote control be established, making remote control operations cumbersome.

[0040] In an embodiment of the present application, during an audio or video call between a first terminal and a second terminal, the first terminal shares the screen with the second terminal, and upon receiving voice data transmitted by the second terminal, the first terminal recognizes the voice data to generate an operation execution instruction, thereby enabling the operation indicated by the operation execution instruction to be executed on the current interface of the first terminal based on the voice signal on the second terminal side, that is, remote control of the first terminal by the second terminal is achieved. Compared with remote control based on screen triggering operations, the remote control method proposed in this application can simplify the remote control process and improve the efficiency of remote control.

[0041] The remote control method proposed in this application will be described below through specific embodiments.

[0042] Please refer to Figure 1 , which shows a schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present application, wherein the computer system includes a first terminal 110 and a second terminal 120. The first terminal 110 and the second terminal 120 perform data communication via a communication network. Optionally, the communication network can be a wired network or a wireless network, and the communication network can be at least one of a local area network, a metropolitan area network, and a wide area network.

[0043] The first terminal 110 is an electronic device with a screen sharing function. The screen sharing function can be a function of a native application in the first terminal 110 or a function of a third-party application. The first terminal 110 can be a smartphone, tablet computer, laptop computer, desktop computer, smart TV, wearable device, or in-vehicle terminal, etc. Figure 1 The first terminal 110 is taken as an example as a smart phone, but this is not limiting.

[0044] The second terminal 120 is an electronic device with a screen sharing function, which can be a function of a native application in the second terminal 120, or a function of a third-party application; the second terminal 120 can be a smart phone, tablet computer, laptop computer, desktop computer, smart TV, wearable device or vehicle-mounted terminal, etc. Figure 1 The second terminal 120 is taken as an example of a smart phone, but this is not limiting.

[0045] In some embodiments, data interaction occurs between the first terminal 110 and the second terminal 120. Figure 1 As shown, during an audio or video call between a first terminal 110 and a second terminal 120, the first terminal 110 and the second terminal 120 transmit audio or video data to each other, and the first terminal 110 sends screen data to the second terminal 120 to share the screen with the second terminal 120. Furthermore, upon receiving the voice data transmitted by the second terminal 120, the first terminal 110 recognizes the voice data and generates an operation execution instruction, thereby executing the operation indicated by the operation execution instruction in the current interface according to the operation execution instruction.

[0046] Please refer to Figure 2 , which shows a flow chart of a remote control method provided by an exemplary embodiment of the present application. This embodiment is described using the method for a first terminal as an example. The method may include the following steps:

[0047] Step 210: Share the screen with the second terminal during the audio or video call with the second terminal.

[0048] In the embodiment of the present application, the first terminal is the controlled terminal, and the second terminal is the controlling terminal. Optionally, the audio and video call between the first terminal and the second terminal can be implemented based on high-definition video call technology, multimedia call technology, or terminal cellular call technology, which is not limited in the embodiment of the present application.

[0049] Optionally, the first terminal is the controlled terminal, and the first terminal can also be an Internet of Things device, which is a device with audio and video call functions, so that the Internet of Things device can control the smart home by establishing remote control with the second terminal.

[0050] Optionally, during an audio and video call between the first terminal and the second terminal, the first terminal and the second terminal respectively collect audio and video data through their respective microphones and cameras, and encode and compress the audio and video data, thereby transmitting the encoded and compressed audio and video data to the other terminal through the network.

[0051] Unlike related technologies, when remote control is required between a first terminal and a second terminal, the user on the first terminal needs to perform corresponding screen triggering operations, such as turning on the remote control switch and agreeing to access, which makes the establishment of remote control complicated. In the embodiments of the present application, the remote control process is linked to the audio and video call process. During an audio and video call between the first and second terminals, the first terminal can achieve remote control between the first and second terminals by sharing the screen image with the second terminal.

[0052] Optionally, screen sharing refers to the process of transmitting the currently displayed screen content of the first terminal to the second terminal in real time, and the second terminal presenting it to the user. In one possible implementation, the first terminal can capture image data of the current screen through screen recording, and by sending the image data to the second terminal, the second terminal generates the screen corresponding to the first terminal based on the image data, thereby achieving screen sharing from the first terminal to the second terminal.

[0053] Indicative, such as Figure 3 As shown, during the audio and video call with the second terminal 120 , the first terminal 110 shares the screen with the second terminal 120 , so that the second terminal 120 displays the screen corresponding to the first terminal 110 .

[0054] Step 220: When receiving the voice data transmitted by the second terminal, recognize the voice data and generate an operation execution instruction.

[0055] In some embodiments, after sharing the screen image with the second terminal, the first terminal continues to receive voice data transmitted by the second terminal, and upon receiving the voice data transmitted by the second terminal, recognizes the voice data through voice recognition technology to obtain a voice recognition result, thereby generating an operation execution instruction based on the voice recognition result.

[0056] That is, in the embodiment of the present application, the first terminal recognizes the voice data transmitted by the second terminal during an audio or video call with the second terminal to generate an operation execution instruction, thereby enabling the second terminal to remotely control the first terminal.

[0057] In one possible implementation, the second terminal collects local voice signals, encodes and compresses the collected voice signals to generate voice data, and then transmits the voice data to the first terminal. After receiving the voice data, the first terminal decodes the voice data to obtain a voice signal. The first terminal then uses voice recognition technology to identify the voice signal and generates an action execution instruction based on the voice recognition result. For example, the voice signal might be, "Xiao A, set the alarm for 3 PM."

[0058] Optionally, an operation execution instruction refers to an instruction used to trigger and control the terminal to perform a specific operation or task. Optionally, the operation execution instruction can be a system-level instruction or an application-level instruction, which is not limited in the embodiments of the present application. For example, for the above voice signal, the operation execution instruction can be "Open the clock application and set an alarm for 3 pm."

[0059] Step 230: Based on the operation execution instruction, execute the operation indicated by the operation execution instruction in the current interface.

[0060] Furthermore, after determining the operation execution instruction, the first terminal can execute the operation indicated by the operation execution instruction in the current interface, wherein the current interface is the interface currently displayed by the first terminal, and the current interface is dynamically updated based on the progress of the operation execution.

[0061] In a possible implementation, after determining the operation execution instruction, the first terminal may call the corresponding functional module or application through a specific interface to perform the operation indicated by the operation execution instruction, such as opening an application, sending a text message, etc.

[0062] For example, for the operation execution instruction of "open the clock application and set the alarm for 3 pm", the first terminal can call the relevant functional modules of the clock application through the system interface, such as calling the alarm manager interface, so as to set the alarm time, repetition cycle, ringtone and other parameters according to the instruction through the functional modules of the clock application, and save the setting information to the system database, so that the alarm will trigger a reminder at the specified time.

[0063] Indicative, such as Figure 3 As shown, when receiving voice data transmitted by the second terminal 120, the first terminal 110 recognizes the voice data and generates an operation execution instruction. Taking the operation execution instruction as "open XX application" as an example, the first terminal 110 executes the operation indicated by the operation execution instruction, opens XX application, and displays the application interface 301 of XX application.

[0064] In summary, in an embodiment of the present application, during an audio or video call between a first terminal and a second terminal, the first terminal shares the screen image with the second terminal, and upon receiving voice data transmitted by the second terminal, the first terminal recognizes the voice data to generate an operation execution instruction, thereby enabling the operation indicated by the operation execution instruction to be performed on the current interface of the first terminal according to the voice signal on the second terminal side, i.e., remote control of the first terminal by the second terminal is achieved. Compared with remote control based on screen triggering operations, the remote control method proposed in this application can simplify the remote control process and improve the efficiency of remote control.

[0065] In some embodiments, considering that the remote control process is combined with the audio and video call process in the embodiments of the present application, in order to avoid affecting the normal audio and video call process, a corresponding instruction for enabling remote control can also be set. Therefore, during the audio and video call with the second terminal, the first terminal will only establish remote control with the second terminal when the specific instruction is present. The above process will be explained below through specific embodiments.

[0066] Please refer to Figure 4 , which shows a flow chart of a remote control method provided by another exemplary embodiment of the present application. This embodiment is described using the method applied to a terminal as an example. The method may include the following steps:

[0067] Step 211: During the audio or video call with the second terminal, a call interface is displayed.

[0068] In a possible implementation, the audio and video call between the first terminal and the second terminal may be initiated by the first terminal or the second terminal.

[0069] Optionally, during the audio call with the second terminal, the call interface displayed by the first terminal may include a call end control, a speaker setting control, etc.

[0070] Optionally, during a video call with a second terminal, the call interface displayed by the first terminal may include not only call end controls, speaker setting controls, etc., but also camera setting controls and video images captured by cameras on both ends.

[0071] Indicative, such as Figure 5 As shown, during an audio or video call with a second terminal, the first terminal displays a call interface 501, which includes a video image captured by the first terminal through a camera and a video image captured by the second terminal through a camera.

[0072] Step 212: Sharing the screen image with the second terminal upon receiving the remote control instruction.

[0073] In a possible implementation, during an audio or video call with a second terminal, the first terminal will share its screen with the second terminal only when receiving a remote control instruction sent by the second terminal, thereby ensuring the quality of the audio or video call.

[0074] The remote control command refers to a command requesting the establishment of remote control. Optionally, the remote control command can be a command generated by voice input during an audio or video call, such as a user on the second terminal inputting the voice command "establish remote control," whereby the second terminal sends the voice remote control command to the first terminal. Alternatively, the remote control command can be a command generated by a screen-triggered operation, such as a remote control control provided in the call interface, whereby, in response to the activation of the remote control control, the second terminal generates a remote control command and sends it to the first terminal.

[0075] Optionally, the remote control command is a voice command. In one possible implementation, during an audio or video call with a second terminal, the first terminal receives voice data transmitted by the second terminal and performs voice recognition on the voice data in real time. When the remote control command is recognized based on the voice data, the first terminal can share its screen with the second terminal.

[0076] Optionally, the second terminal may also request to establish remote control with the first terminal by sending a remote control key to the first terminal. In one possible implementation, the user may store a pair of keys between the first terminal and the second terminal in advance. The key may be a symmetric key or an asymmetric key, which is not limited in this embodiment of the present application. Furthermore, when the second terminal receives the user's operation to turn on the remote control control in the call interface, the second terminal sends the remote control key to the first terminal. When the first terminal receives the remote control key and passes the decryption verification, it can share the screen with the second terminal.

[0077] Optionally, in order to ensure the security of remote control, after receiving the remote control instruction, the first terminal may further authenticate the second terminal first, and then determine whether to share the screen with the second terminal according to the authority authentication result.

[0078] In one possible implementation, upon receiving a remote control command, the first terminal may perform an authorization authentication on the second terminal based on the audio and video call data between the first terminal and the second terminal, obtain an authorization authentication result, and determine whether to share the screen with the second terminal based on the authorization authentication result. Optionally, if the authorization authentication result indicates that the authentication is successful, the first terminal shares the screen with the second terminal; if the authorization authentication result indicates that the authentication is unsuccessful, the first terminal does not share the screen with the second terminal.

[0079] Optionally, the first terminal may obtain authentication information from the audio and video call data between the first terminal and the second terminal, thereby performing authorization authentication on the second terminal. In one possible embodiment, upon receiving a remote control command, the first terminal determines at least one of the call account information and the biometric information of the call recipient based on the audio and video call data between the first terminal and the second terminal, thereby performing authorization authentication on the second terminal based on at least one of the call account information and the biometric information of the call recipient, and obtaining an authorization authentication result.

[0080] Optionally, the call account information may include at least one of the information associated with the account, such as the account name, account ID (Identification), account IP (Internet Protocol) address, etc., which is not limited in this embodiment of the present application.

[0081] Optionally, biometric information may include facial information, voiceprint information, palmprint information, iris information, and other information that can be obtained through audio and video calls, which is not limited in this embodiment of the present application. Facial information can be obtained by performing facial recognition on the video footage in the audio and video call data, voiceprint information can be obtained by performing voiceprint recognition on the voice data in the audio and video call data, palmprint information can be obtained by performing palmprint recognition on the video footage in the audio and video call data, and iris information can be obtained by performing iris recognition on the video footage in the audio and video call data. This embodiment of the present application does not limit the specific recognition technology used.

[0082] Optionally, to authenticate the second terminal's permissions, the first terminal may also pre-store a master permission list, which includes the permitted account information and permitted biometric information of those with control permissions. For example, if the first terminal is held by an elder or child, the user may pre-store their account information and / or biometric information in the master permission list of the first terminal.

[0083] In one possible implementation, upon receiving a remote control instruction, the first terminal queries the master control authority list based on at least one of the call account information and the object's biometric information, and matches the call account information with the permitted account information pre-stored in the master control authority list, and matches the object's biometric information with the permitted biometric information pre-stored in the master control authority list. The first terminal can then determine the authority authentication result based on at least one of the matching between the call account information and the permitted account information, and the matching between the object's biometric information and the permitted biometric information.

[0084] Optionally, when only the call account information is determined, the first terminal determines the permission authentication result based on whether the call account information matches the permission account information. If the permission account information matches the call account information, the permission authentication is passed; if the permission account information does not match the call account information, the permission authentication fails.

[0085] Optionally, when only the subject's biometric information is determined, the first terminal determines the authorization verification result based on whether the subject's biometric information matches the permitted biometric information. If the permitted biometric information matches the subject's biometric information, the authorization verification is successful; if not, the authorization verification is unsuccessful.

[0086] Optionally, after determining the call account information and the subject's biometric information, the first terminal may determine the result of the authorization authentication based on the matching of the call account information with the permitted account information, and the matching of the subject's biometric information with the permitted biometric information. If the permitted account information matches the call account information, and the permitted biometric information matches the subject's biometric information, the authorization authentication is successful. If the permitted account information does not match the call account information, or the permitted biometric information does not match the subject's biometric information, the authorization authentication is unsuccessful.

[0087] The authorized account information and authorized biometric information of the same authorized user can be stored in association in the master permission list. The authorization authentication is considered successful only when the authorized account information matches the call account information, the authorized biometric information matches the target biometric information, and the authorized account information and authorized biometric information are stored in an associated state.

[0088] In one possible implementation, in order to facilitate the second terminal to perform remote control after sharing the screen, upon receiving a remote control instruction, the first terminal can also first switch the call interface to the main interface, and then share the screen to the second terminal, so that the user on the second terminal side can directly perform voice input operations based on the displayed screen containing the main interface of the first terminal.

[0089] Indicative, such as Figure 5 As shown, upon receiving a remote control command, the first terminal switches the call interface 501 to the main interface 502 and shares the screen with the second terminal. Furthermore, upon receiving voice data transmitted by the second terminal, the first terminal recognizes the voice data and generates an operation execution instruction. For example, if the operation execution instruction is "Open XX application," the first terminal executes the operation indicated by the operation execution instruction, opens XX application, and displays the application interface 503 of XX application.

[0090] Accordingly, when remote control ends, upon receiving a remote end command from the second terminal, the first terminal can stop sharing its screen with the second terminal. Furthermore, to facilitate the user's ability to perform regular audio and video call operations in the call interface, the first terminal can switch the current interface back to the call interface to restore the interface display effect during a normal audio and video call.

[0091] The remote end command refers to a command requesting the disconnection of remote control. Optionally, the remote end command can be a command generated by voice input during an audio or video call, such as a user on the second terminal inputting the voice command "disconnect remote control," whereby the second terminal sends the voice remote end command to the first terminal. Alternatively, the remote end command can be a command generated by a screen-triggered operation, such as a remote control control provided in the call interface, whereby the second terminal generates a remote end command and sends it to the first terminal in response to the closing operation of the remote control control.

[0092] Indicative, such as Figure 5 As shown, when the remote control ends, upon receiving the remote end instruction sent by the second terminal, the first terminal stops sharing the screen with the second terminal and switches the current interface (the application interface 503 of the XX application) back to the call interface 501.

[0093] Optionally, the remote end instruction is a voice instruction. In one possible implementation, during an audio or video call with a second terminal, the first terminal receives voice data transmitted by the second terminal and performs voice recognition on the voice data in real time, so that when the remote end instruction is recognized based on the voice data, the first terminal stops sharing the screen with the second terminal.

[0094] Optionally, the second terminal may also request to disconnect remote control from the first terminal by sending a remote end key to the first terminal. In one possible implementation, the user may store a pair of keys between the first terminal and the second terminal in advance. The key may be a symmetric key or an asymmetric key, which is not limited in this embodiment of the present application. Furthermore, when the second terminal receives a user's closing operation on the remote control control in the call interface, the second terminal sends a remote end key to the first terminal. When the first terminal receives the remote end key and passes the decryption verification, it can stop sharing the screen with the second terminal.

[0095] Step 221: When receiving voice data transmitted by the second terminal, recognize the voice data to obtain a voice recognition result.

[0096] In one possible implementation, upon receiving voice data transmitted by the second terminal, the first terminal may first recognize the voice data to obtain a voice recognition result in order to generate a corresponding operation execution instruction. Alternatively, the voice recognition process may be implemented based on voice recognition technology, such as by recognizing the voice data using a voice recognition model to obtain a voice recognition result.

[0097] Step 222 : When the speech recognition result indicates that there is a control intention, generate an operation execution instruction based on the speech recognition result.

[0098] In one possible implementation, after establishing remote control with the second terminal, considering that the first terminal and the second terminal are still in an audio and video call, that is, the voice signal input by the user on the second terminal side may not be used to implement remote control, therefore, in order to improve the accuracy of remote control, the first terminal also needs to first determine whether the voice signal is used to implement remote control.

[0099] Optionally, the first terminal can analyze whether the voice data has a control intention based on the voice recognition result, so that only when the voice recognition result indicates that there is a control intention, the first terminal will generate an operation execution instruction based on the voice recognition result; if the voice recognition result indicates that there is no control intention, no operation execution instruction will be generated.

[0100] Optionally, the first terminal may input the speech recognition result into the intention recognition model, and output the intention recognition result through the intention recognition model, so that when it is determined that there is a control intention, the first terminal generates an operation execution instruction based on the speech recognition result and the intention recognition result.

[0101] In some embodiments, the process of performing speech recognition on the first terminal side can be implemented through a voice control function. Optionally, the voice control function can be a function of a system application in the first terminal or a function of a third-party application, which is not limited in this embodiment of the present application. For example, the voice control function is a function of a voice assistant application.

[0102] In one possible implementation, during an audio or video call with a second terminal, in order to promptly respond to the remote control of the second terminal, the first terminal turns on a voice control function, and the voice control function supports recognizing call voices. Then, when the first terminal receives voice data transmitted by the second terminal, the first terminal can recognize the voice data through the voice control function and generate operation execution instructions.

[0103] For example, during an audio or video call with a second terminal, the first terminal opens a voice assistant application, and upon receiving voice data transmitted by the second terminal, transmits the voice data as input to the voice assistant application, thereby recognizing the voice data through the voice control function within the voice assistant application and generating operation execution instructions.

[0104] Step 230: Based on the operation execution instruction, execute the operation indicated by the operation execution instruction in the current interface.

[0105] The implementation of this step can refer to the above embodiment, and this embodiment will not be described in detail here.

[0106] In the above embodiment, during an audio or video call with a second terminal, remote control is established and terminated by recognizing remote control commands and remote termination commands. Furthermore, during the remote control establishment process, voice data sent by the second terminal is directly recognized to generate operation execution commands, thereby simplifying the implementation of remote control and improving its efficiency.

[0107] In addition, after receiving the remote control instruction, the security of the remote control is ensured by further performing authority authentication on the second terminal and sharing the screen image with the second terminal only when the authority authentication passes.

[0108] Please refer to Figure 6 , which shows a flow chart of a remote control method provided by another exemplary embodiment of the present application. This embodiment is described using the method for the second terminal as an example. The method may include the following steps:

[0109] Step 610: During the audio or video call with the first terminal, a screen image corresponding to the first terminal is displayed.

[0110] The first terminal is the controlled terminal, and the second terminal is the controlling terminal. Optionally, the audio and video call between the first terminal and the second terminal can be implemented based on high-definition video call technology, multimedia call technology, or terminal cellular call technology, which is not limited in this embodiment of the application.

[0111] In one possible implementation, the first terminal may capture image data of the current screen by screen recording, and send the image data to the second terminal so that the second terminal can decode the image data after receiving the image data, thereby generating and displaying the screen corresponding to the first terminal.

[0112] Step 620: Transmit the collected voice data to the first terminal so that the first terminal recognizes the voice data and generates an operation execution instruction.

[0113] When the second terminal displays the screen image corresponding to the first terminal, the second terminal collects the voice data input by the user in real time and transmits the collected voice data to the first terminal so that the first terminal can recognize the voice data and generate an operation execution instruction.

[0114] In one possible implementation, the second terminal collects a local voice signal, encodes and compresses the collected voice signal to generate voice data, and then transmits the voice data to the first terminal. After receiving the voice data, the first terminal decodes the voice data to obtain a voice signal, recognizes the voice signal using voice recognition technology, and generates an operation execution instruction based on the voice recognition result.

[0115] Optionally, the operation execution instruction refers to an instruction for triggering and controlling the terminal to perform a specific operation or task. Optionally, the operation execution instruction can be a system-level instruction or an application-level instruction, which is not limited in the embodiment of the present application.

[0116] Step 630: After the first terminal executes the operation indicated by the operation execution instruction, the screen corresponding to the first terminal is updated and displayed.

[0117] Furthermore, after determining the operation execution instruction, the first terminal can execute the operation indicated by the operation execution instruction in the current interface, and in the process of executing the instruction operation, the first terminal records the screen image in real time and sends the recorded image data to the second terminal, so that the second terminal can receive the image data and update the display of the screen image corresponding to the first terminal, so that the user on the second terminal side can view the screen changes in the process of the first terminal responding to remote control.

[0118] In summary, in an embodiment of the present application, during an audio or video call between a first terminal and a second terminal, the first terminal shares the screen image with the second terminal, and upon receiving voice data transmitted by the second terminal, the first terminal recognizes the voice data to generate an operation execution instruction, thereby enabling the operation indicated by the operation execution instruction to be performed on the current interface of the first terminal according to the voice signal on the second terminal side, i.e., remote control of the first terminal by the second terminal is achieved. Compared with remote control based on screen triggering operations, the remote control method proposed in this application can simplify the remote control process and improve the efficiency of remote control.

[0119] In some embodiments, considering that the remote control process is combined with the audio and video call process in the embodiments of the present application, in order to avoid affecting the normal audio and video call process, corresponding instructions can also be set for turning on remote control, so that during the audio and video call with the first terminal, the second terminal can only establish remote control with the first terminal when specific instructions are present.

[0120] In one possible implementation, an audio or video call between a first terminal and a second terminal can be initiated by either the first terminal or the second terminal. During the audio or video call with the first terminal, the second terminal first displays the call interface. Then, in response to the user's input, the second terminal sends the received remote control command to the first terminal. Furthermore, when the first terminal receives the remote control command and shares its screen with the second terminal, the second terminal can display the screen corresponding to the first terminal.

[0121] Optionally, the user can input commands by inputting voice signals, that is, the remote control commands are commands in the form of voice. In one possible implementation, the second terminal collects the voice signals input by the user in real time, generates voice data, and sends it to the first terminal, which then identifies whether the voice data is a remote control command.

[0122] Optionally, the user can also input commands by triggering an operation through a control, i.e., the remote control command is a command in the form of operation data. In one possible embodiment, the call interface is provided with a remote control control, so that in response to the user triggering the remote control command, the second terminal sends the operation data as a remote control command to the first terminal.

[0123] In another possible embodiment, during an audio or video call with the first terminal and establishing remote control, in response to the user's command input operation, the second terminal can also send the received remote end command to the first terminal to cause the first terminal to stop sharing the screen. In this way, when the first terminal stops sharing the screen to the second terminal, in order to facilitate the user to perform regular audio or video call operations in the call interface, the second terminal can also redisplay the call interface.

[0124] In the above embodiment, remote control is established during the audio and video call with the first terminal, and the corresponding screen is switched and displayed in combination with the remote control process, so that the user's audio and video call experience can be guaranteed before and after the remote control is established.

[0125] Please refer to Figure 7 , which shows a schematic diagram of remote control interaction provided by an exemplary embodiment of the present application. Take the first terminal 710 supporting audio and video calls and having voice control functions and voiceprint recognition functions, and the second terminal 720 supporting audio and video calls as an example.

[0126] Indicative, such as Figure 7As shown, during an audio and video call between a first terminal 710 and a second terminal 720, when the first terminal 710 receives voice data sent by the second terminal 720 and recognizes a remote control instruction based on the voice data, the first terminal 710 performs an authorization authentication on the second terminal 720 based on the audio and video call data between the first terminal 710 and the second terminal 720. Thus, if the authorization authentication is passed, the first terminal 710 shares the screen with the second terminal 720 and continues to receive voice data sent by the second terminal 720.

[0127] Furthermore, when receiving voice data sent by the second terminal 720, the first terminal 710 recognizes the voice data through a voice assistant application with a voice control function and generates an operation execution instruction, so that the first terminal 710 can perform the operation indicated by the operation execution instruction based on the operation execution instruction.

[0128] Please refer to Figure 8 , which shows an interactive schematic diagram of remote control provided by another exemplary embodiment of the present application.

[0129] Indicative, such as Figure 8 As shown, a first terminal 810 and a second terminal 820 establish an audio or video call. This audio or video call can be initiated by the first terminal 810 as the caller, or by the second terminal 820 as the caller. During the audio or video call between the first terminal 810 and the second terminal 820, the second terminal 820 receives the user's voice input and sends the collected voice data to the first terminal 810 to initiate remote control of the first terminal 810. Upon recognizing the remote control command, the first terminal 810 obtains the call account information and biometric information of the second terminal 820 based on the audio or video call data between the first terminal 810 and the second terminal 820, and performs an authorization authentication on the second terminal 820. If the authorization authentication passes, the first terminal 810 establishes remote control with the second terminal 820 and shares the screen with the second terminal 820. Simultaneously, the first terminal 810 launches a voice assistant application with voice control functionality and continues to receive voice data sent by the second terminal 820.

[0130] Then, when the voice data sent by the second terminal 820 is received, the first terminal 810 recognizes the voice data through the voice assistant application and generates an operation execution instruction, so that the first terminal 810 can perform the operation indicated by the operation execution instruction based on the operation execution instruction.

[0131] Furthermore, when the first terminal 810 recognizes the remote end command, the first terminal 810 turns off the remote control between the second terminal 820 and stops sharing the screen with the second terminal 820, so that both the first terminal 810 and the second terminal 820 resume displaying the call interface and continue the audio and video call.

[0132] Please refer to Figure 9 , which shows a block diagram of the structure of a remote control device provided by an exemplary embodiment of the present application. The device includes:

[0133] The screen sharing module 901 is used to share the screen with the second terminal during an audio or video call with the second terminal;

[0134] The instruction generation module 902 is configured to, upon receiving voice data transmitted by the second terminal, identify the voice data and generate an operation execution instruction;

[0135] The operation execution module 903 is used to execute the operation indicated by the operation execution instruction in the current interface based on the operation execution instruction.

[0136] Optionally, the screen sharing module 901 includes:

[0137] An interface display unit, configured to display a call interface during an audio or video call with the second terminal;

[0138] The screen sharing unit is configured to share the screen image with the second terminal upon receiving a remote control instruction.

[0139] Optionally, the screen sharing unit is configured to:

[0140] Upon receiving the remote control instruction, performing authority authentication on the second terminal based on the audio and video call data between the second terminal and the second terminal to obtain an authority authentication result;

[0141] When the authority identification result indicates that the authentication is passed, the screen image is shared with the second terminal.

[0142] Optionally, the screen sharing unit is configured to:

[0143] Upon receiving the remote control command, determining at least one of call account information and biometric information of a call partner based on the audio and video call data with the second terminal;

[0144] Based on at least one of the call account information and the target biometric information, authority authentication is performed on the second terminal to obtain the authority authentication result.

[0145] Optionally, the screen sharing unit is configured to:

[0146] Based on at least one of the call account information and the subject's biometric information, querying a master control authority list, the master control authority list including permitted account information and permitted biometric information having control authority;

[0147] The authority authentication result is determined based on at least one of a matching condition between the call account information and the permitted account information, and a matching condition between the target biometric information and the permitted biometric information.

[0148] Optionally, the screen sharing unit is configured to:

[0149] When the remote control command is received, the call interface is switched to the main interface, and the screen image is shared with the second terminal;

[0150] The device further comprises:

[0151] The screen sharing module 901 is configured to stop sharing the screen to the second terminal and display the call interface when a remote end instruction is received.

[0152] Optionally, the screen sharing unit is configured to:

[0153] During the audio or video call with the second terminal, receiving voice data transmitted by the second terminal;

[0154] When the remote control command is recognized based on the voice data, the screen image is shared with the second terminal.

[0155] Optionally, the instruction generation module 902 is configured to:

[0156] When receiving the voice data transmitted by the second terminal, recognizing the voice data to obtain a voice recognition result;

[0157] In a case where the voice recognition result indicates that there is a control intention, the operation execution instruction is generated based on the voice recognition result.

[0158] Optionally, the device further includes:

[0159] A function activation module, configured to activate a voice control function during an audio or video call with the second terminal, wherein the voice control function supports recognition of call voice;

[0160] The instruction generation module 902 is used to:

[0161] When the voice data transmitted by the second terminal is received, the voice data is recognized by the voice control function, and the operation execution instruction is generated.

[0162] In summary, in an embodiment of the present application, during an audio or video call between a first terminal and a second terminal, the first terminal shares the screen image with the second terminal, and upon receiving voice data transmitted by the second terminal, the first terminal recognizes the voice data to generate an operation execution instruction, thereby enabling the operation indicated by the operation execution instruction to be performed on the current interface of the first terminal according to the voice signal on the second terminal side, i.e., remote control of the first terminal by the second terminal is achieved. Compared with remote control based on screen triggering operations, the remote control method proposed in this application can simplify the remote control process and improve the efficiency of remote control.

[0163] Please refer to Figure 10 , which shows a structural block diagram of a remote control device provided by another exemplary embodiment of the present application. The device includes:

[0164] The screen display module 1001 is used to display the screen corresponding to the first terminal during the audio and video call with the first terminal;

[0165] a data transmission module 1002, configured to transmit the collected voice data to the first terminal, so that the first terminal recognizes the voice data and generates an operation execution instruction;

[0166] The screen updating module 1003 is configured to update and display a screen corresponding to the first terminal after the first terminal executes the operation indicated by the operation execution instruction.

[0167] Optionally, the screen display module 1001 is used to:

[0168] During the audio or video call with the first terminal, displaying a call interface and sending a remote control command to the first terminal;

[0169] When the first terminal receives the remote control instruction and shares the screen image with the second terminal, the screen image corresponding to the first terminal is displayed.

[0170] Optionally, the device further includes:

[0171] an instruction sending module, configured to send a remote end instruction to the first terminal, so that the first terminal stops sharing the screen image;

[0172] The interface display module is used to redisplay the call interface when the first terminal stops sharing the screen.

[0173] In summary, in an embodiment of the present application, during an audio or video call between a first terminal and a second terminal, the first terminal shares the screen image with the second terminal, and upon receiving voice data transmitted by the second terminal, the first terminal recognizes the voice data to generate an operation execution instruction, thereby enabling the operation indicated by the operation execution instruction to be performed on the current interface of the first terminal according to the voice signal on the second terminal side, i.e., remote control of the first terminal by the second terminal is achieved. Compared with remote control based on screen triggering operations, the remote control method proposed in this application can simplify the remote control process and improve the efficiency of remote control.

[0174] It should be noted that the apparatus provided in the above embodiments is merely exemplified by the division of the above functional modules. In actual applications, the above functions can be distributed among different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The implementation process is detailed in the method embodiments and will not be repeated here.

[0175] It should be noted that before obtaining relevant user data and during the process of obtaining relevant user data, this application can display a prompt interface, pop-up window or output voice prompt information. The prompt interface, pop-up window or voice prompt information is used to remind the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining user-related data after obtaining the user's confirmation operation on the prompt interface or pop-up window. Otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining user-related data are terminated, that is, the user's relevant data is not obtained. In other words, the information involved in this application (including but not limited to user device information, user personal information, etc., user's corresponding operation data), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0176] See also Figure 11 , Figure 11 FIG1 is a schematic diagram of a terminal structure provided by an exemplary embodiment of the present application. The terminal may further include one or more of the following components: a processor 1110 and a memory 1120.

[0177] Optionally, the processor 1110 utilizes various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1120, and calling data stored in the memory 1120. Optionally, the processor 1110 can be implemented in the form of at least one hardware of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA).

[0178] The processor 1110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), and a baseband chip. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch screen; the NPU is used to implement artificial intelligence (AI) functions; and the baseband chip is used to process wireless communications. It is understandable that the above-mentioned baseband chip may not be integrated into the processor 1110, but may be implemented separately through a single chip.

[0179] The memory 1120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1120 includes a non-transitory computer-readable storage medium. The memory 1120 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0180] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more (such as power components, display components, sensor components) or fewer components than shown in the figure, or a combination of certain components, or different component arrangements.

[0181] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the remote control method as described in the above embodiment.

[0182] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the remote control method described in the above embodiment.

[0183] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0184] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A remote control method, characterized in that: The method is used for a first terminal, and includes: During an audio or video call with a second terminal, sharing the screen with the second terminal; When receiving voice data transmitted by the second terminal, recognizing the voice data and generating an operation execution instruction; Based on the operation execution instruction, the operation indicated by the operation execution instruction is executed in the current interface.

2. The method according to claim 1, characterized in that The step of sharing the screen image with the second terminal during the audio or video call with the second terminal includes: During the audio or video call with the second terminal, displaying a call interface; When a remote control instruction is received, the screen image is shared with the second terminal.

3. The method according to claim 2, characterized in that Sharing the screen image to the second terminal upon receiving the remote control instruction includes: Upon receiving the remote control instruction, performing authority authentication on the second terminal based on the audio and video call data between the second terminal and the second terminal to obtain an authority authentication result; When the authority identification result indicates that the authentication is passed, the screen image is shared with the second terminal.

4. The method according to claim 3, characterized in that The method of performing, upon receiving the remote control instruction, authorization authentication on the second terminal based on the audio and video call data between the second terminal and the second terminal, and obtaining an authorization authentication result includes: Upon receiving the remote control command, determining at least one of call account information and biometric information of a call partner based on the audio and video call data with the second terminal; Based on at least one of the call account information and the target biometric information, authority authentication is performed on the second terminal to obtain the authority authentication result.

5. The method according to claim 4, characterized in that The performing authority authentication on the second terminal based on at least one of the call account information and the target biometric information to obtain the authority authentication result includes: Based on at least one of the call account information and the subject's biometric information, querying a master control authority list, the master control authority list including permitted account information and permitted biometric information having control authority; The authority authentication result is determined based on at least one of a matching condition between the call account information and the permitted account information, and a matching condition between the target biometric information and the permitted biometric information.

6. The method according to claim 2, characterized in that Sharing the screen image to the second terminal upon receiving the remote control instruction includes: When the remote control command is received, the call interface is switched to the main interface, and the screen image is shared with the second terminal; The method further comprises: When a remote end instruction is received, the screen image is stopped from being shared with the second terminal, and the call interface is displayed.

7. The method according to claim 2, characterized in that Sharing the screen image to the second terminal upon receiving the remote control instruction includes: During the audio or video call with the second terminal, receiving voice data transmitted by the second terminal; When the remote control command is recognized based on the voice data, the screen image is shared with the second terminal.

8. The method according to any one of claims 1 to 7, characterized in that: The step of, upon receiving voice data transmitted by the second terminal, recognizing the voice data and generating an operation execution instruction, includes: When receiving the voice data transmitted by the second terminal, recognizing the voice data to obtain a voice recognition result; In a case where the voice recognition result indicates that there is a control intention, the operation execution instruction is generated based on the voice recognition result.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: During the audio or video call with the second terminal, turning on the voice control function, wherein the voice control function supports recognizing the call voice; The step of, upon receiving voice data transmitted by the second terminal, recognizing the voice data and generating an operation execution instruction, includes: When the voice data transmitted by the second terminal is received, the voice data is recognized by the voice control function, and the operation execution instruction is generated.

10. A remote control method, characterized in that: The method is used for a second terminal, and the method includes: During an audio or video call with a first terminal, displaying a screen corresponding to the first terminal; transmitting the collected voice data to the first terminal so that the first terminal recognizes the voice data and generates an operation execution instruction; After the first terminal executes the operation indicated by the operation execution instruction, the screen image corresponding to the first terminal is updated and displayed.

11. The method according to claim 10, characterized in that The step of displaying a screen corresponding to the first terminal during the audio or video call with the first terminal includes: During the audio or video call with the first terminal, displaying a call interface and sending a remote control command to the first terminal; When the first terminal receives the remote control instruction and shares the screen image with the second terminal, the screen image corresponding to the first terminal is displayed.

12. The method according to claim 10, characterized in that The method further comprises: Sending a remote end instruction to the first terminal to cause the first terminal to stop sharing the screen; When the first terminal stops sharing the screen, the call interface is redisplayed.

13. A remote control device, characterized in that: The device comprises: A screen sharing module, configured to share a screen image with a second terminal during an audio or video call with the second terminal; an instruction generating module, configured to, upon receiving voice data transmitted by the second terminal, identify the voice data and generate an operation execution instruction; The operation execution module is used to execute the operation indicated by the operation execution instruction in the current interface based on the operation execution instruction.

14. A remote control device, characterized in that: The device comprises: A screen display module, configured to display a screen corresponding to the first terminal during an audio or video call with the first terminal; a data transmission module, configured to transmit the collected voice data to the first terminal, so that the first terminal recognizes the voice data and generates an operation execution instruction; The screen updating module is configured to update and display a screen corresponding to the first terminal after the first terminal executes the operation indicated by the operation execution instruction.

15. A terminal, characterized in that: The terminal includes a processor and a memory; the memory stores at least one computer instruction, and the at least one computer instruction is used to be executed by the processor to implement the remote control method according to any one of claims 1 to 9, or the remote control method according to any one of claims 10 to 12.

16. A computer-readable storage medium, characterized in that The storage medium stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the remote control method according to any one of claims 1 to 9, or the remote control method according to any one of claims 10 to 12.

17. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; the processor of the terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal implements the remote control method according to any one of claims 1 to 9, or the remote control method according to any one of claims 10 to 12.

Images