Remote assistance method and system for mobile terminal, storage medium, computer device

Abstract

This application discloses a remote assistance method and system for a mobile terminal, a storage medium, and a computer device. The method includes: a first mobile terminal responding to a remote assistance trigger operation in a remote assistance application; the first mobile terminal sending a remote assistance request carrying a second mobile terminal identifier to a server to accurately establish a connection with the second mobile terminal; the remote assistance application in the first mobile terminal activating the screen recording function of the first terminal, recording the screen information requiring assistance and sending it to the second mobile terminal through the server; the second mobile terminal generating an assistance operation instruction based on the recorded screen information and sending it to the first mobile terminal through the server; and the first mobile terminal performing the corresponding remote assistance operation according to the assistance operation instruction. This enables the second mobile terminal to remotely operate the first mobile terminal, and the remote assistance operation process can be displayed on the first mobile terminal, providing convenience for the user of the first mobile terminal.

Description

Technical Field

[0001] This application relates to the field of remote control technology, and in particular to a remote assistance method and system for a mobile terminal, a storage medium, and a computer device. Background Technology

[0002] With the development of mobile communication technology and the improvement of people's living standards, the use of mobile terminals is becoming increasingly widespread. Mobile terminals have become an indispensable communication tool in people's daily lives. However, when the users of mobile terminals are elderly people or children, these users are not very proficient in using mobile terminals and are unfamiliar with many of their functions, which necessitates remote assistance. Therefore, how to propose a convenient method for remote assistance using mobile terminals has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, this application provides a remote assistance method and system for mobile terminals, a storage medium, and a computer device, which helps to provide convenience for mobile terminal users.

[0004] According to one aspect of this application, a remote assistance method for a mobile terminal is provided, the method comprising:

[0005] In response to a remote assistance trigger operation in a remote assistance application, the first mobile terminal sends a remote assistance request to the server, wherein the remote assistance request carries the identifier of the second mobile terminal.

[0006] The server forwards the remote assistance request to the second mobile terminal, and upon receiving the assistance confirmation feedback sent by the second mobile terminal, forwards the assistance confirmation feedback to the first mobile terminal.

[0007] The first mobile terminal initiates the screen recording function based on the remote assistance application and sends the recorded screen display data to the second mobile terminal through the server;

[0008] The second mobile terminal displays data based on the screen display data and sends assistance operation instructions to the server based on the operation data of the display interface.

[0009] The server sends the assistance operation instruction to the first mobile terminal, so that the first mobile terminal performs the corresponding remote assistance operation based on the assistance operation instruction.

[0010] Optionally, the remote assistance request includes an assistance question; the method further includes:

[0011] The server generates a remote assistance recording based on the assistance question and the screen display data, and stores the remote assistance recording.

[0012] Accordingly, the server forwards the remote assistance request to the second mobile terminal, including:

[0013] The server queries whether the stored remote assistance recordings contain a target remote assistance recording that matches the assistance question;

[0014] If included, the target remote assistance video is sent to the first mobile terminal, and upon receiving the first continued assistance request from the first mobile terminal, the remote assistance request is forwarded to the second mobile terminal;

[0015] If not included, the remote assistance request is forwarded to the second mobile terminal.

[0016] Optionally, the method further includes:

[0017] The first mobile terminal receives the target remote assistance video and outputs a first query message to ask the first mobile terminal user whether to view the historical video that matches the currently requested assistance question.

[0018] If a signal confirming the viewing of historical recordings is received, the target remote assistance recording is played. After the playback ends, a second query message is output to ask the first mobile terminal user whether the current requested assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, the first request to continue assistance is sent to the server.

[0019] If a signal is received indicating that viewing historical recordings is refused, the first request for continued assistance is sent to the server.

[0020] Optionally, the remote assistance request includes an assistance question; the method further includes:

[0021] The server generates operation guidance data corresponding to the assistance question based on the screen display data and the assistance operation instructions, and stores the operation guidance data of the assistance question;

[0022] Accordingly, the server forwards the remote assistance request to the second mobile terminal, including:

[0023] The server queries whether the stored operation guidance data contains target operation guidance data that matches the assistance question.

[0024] If included, the target operation guidance data is sent to the first mobile terminal, and upon receiving the second continued assistance request from the first mobile terminal, the remote assistance request is forwarded to the second mobile terminal.

[0025] If not included, the remote assistance request is forwarded to the second mobile terminal.

[0026] Optionally, the server generates operation guidance data corresponding to the assistance question based on the screen display data and the assistance operation instructions, including:

[0027] The server extracts the operation frame image corresponding to the assistance operation instruction from the screen display data, and determines the operation component corresponding to the operation frame image and the operation method of the operation component based on the operation position and operation method corresponding to the assistance operation instruction.

[0028] The operation components and operation methods are labeled in the operation frame image to obtain an operation guidance image, wherein the operation guidance data includes the operation guidance image.

[0029] Optionally, the method further includes:

[0030] The first mobile terminal receives the target operation guidance data and outputs a third query message to ask the first mobile terminal user whether to view historical operation guidance data that matches the currently requested assistance question;

[0031] If a signal confirming the viewing of historical operation guidance data is received, the target operation guidance data is played. After the playback ends, a fourth query message is output to ask the first mobile terminal user whether the current assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, a second request for continued assistance is sent to the server.

[0032] If a signal is received indicating that viewing historical operation guidance data is refused, a second request for continued assistance is sent to the server.

[0033] Optionally, the method further includes:

[0034] The first mobile terminal sends a target viewing request to the server to query video recordings or operation guidance data;

[0035] The server queries the stored remote assistance video or operation guide data to see if there is any remote assistance video or operation guide data that matches the target's viewing request.

[0036] If it exists, the server sends a matching historical video recording or historical operation guidance request to the first mobile terminal based on the target viewing request, so that the first mobile terminal user can view it;

[0037] If it does not exist, a request failure message is sent to the first mobile terminal, and a fifth query message is output to ask the first mobile terminal user whether a remote assistance request needs to be triggered.

[0038] According to another aspect of this application, a remote assistance system for a mobile terminal is provided, the system comprising:

[0039] First mobile terminal, second mobile terminal, and server;

[0040] The first mobile terminal is configured to send a remote assistance request to the server in response to a remote assistance trigger operation in a remote assistance application, wherein the remote assistance request carries a second mobile terminal identifier.

[0041] The server is configured to forward the remote assistance request to the second mobile terminal, and upon receiving the assistance confirmation feedback sent by the second mobile terminal, forward the assistance confirmation feedback to the first mobile terminal.

[0042] The first mobile terminal is also configured to initiate a screen recording function based on the remote assistance application and send the recorded screen display data to the second mobile terminal through the server;

[0043] The second mobile terminal is used to display data based on the screen display data and to send assistance operation instructions to the server based on operation data of the display interface.

[0044] The server is further configured to send the assistance operation instruction to the first mobile terminal, so that the first mobile terminal performs the corresponding remote assistance operation based on the assistance operation instruction.

[0045] Optionally, the server is also used for:

[0046] A remote assistance recording is generated based on the assistance question and the screen display data, and the remote assistance recording is stored.

[0047] Optionally, the server is also used for:

[0048] Check if the stored remote assistance recordings contain a target remote assistance recording that matches the assistance question;

[0049] If included, the target remote assistance video is sent to the first mobile terminal, and upon receiving the first continued assistance request from the first mobile terminal, the remote assistance request is forwarded to the second mobile terminal;

[0050] If not included, the remote assistance request is forwarded to the second mobile terminal.

[0051] Optionally, the first mobile terminal is further configured to:

[0052] Receive the target remote assistance video and output a first query message to ask the first mobile terminal user whether to view the historical video that matches the currently requested assistance question;

[0053] If a signal confirming the viewing of historical recordings is received, the target remote assistance recording is played. After the playback ends, a second query message is output to ask the first mobile terminal user whether the current requested assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, the first request to continue assistance is sent to the server.

[0054] If a signal is received indicating that viewing historical recordings is refused, the first request for continued assistance is sent to the server.

[0055] Optionally, the server is also used for:

[0056] Based on the screen display data and the assistance operation instructions, operation guidance data corresponding to the assistance question is generated and stored.

[0057] Optionally, the server is also used for:

[0058] Check whether the stored operation guidance data contains target operation guidance data that matches the assistance question;

[0059] If included, the target operation guidance data is sent to the first mobile terminal, and upon receiving the second continued assistance request from the first mobile terminal, the remote assistance request is forwarded to the second mobile terminal.

[0060] If not included, the remote assistance request is forwarded to the second mobile terminal.

[0061] Optionally, the server is also used for:

[0062] Extract the operation frame image corresponding to the assistance operation instruction from the screen display data, and determine the operation component corresponding to the operation frame image and the operation method of the operation component based on the operation position and operation method corresponding to the assistance operation instruction.

[0063] The operation components and operation methods are labeled in the operation frame image to obtain an operation guidance image, wherein the operation guidance data includes the operation guidance image.

[0064] Optionally, the first mobile terminal is further configured to:

[0065] Receive the target operation guidance data and output a third query message to ask the first mobile terminal user whether to view the historical operation guidance data that matches the currently requested assistance question;

[0066] If a signal confirming the viewing of historical operation guidance data is received, the target operation guidance data is played. After the playback ends, a fourth query message is output to ask the first mobile terminal user whether the current assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, a second request for continued assistance is sent to the server.

[0067] If a signal is received indicating that viewing historical operation guidance data is refused, a second request for continued assistance is sent to the server.

[0068] Optionally, the first mobile terminal is further configured to:

[0069] Send a target viewing request to the server to query video recordings or operation guidance data;

[0070] Optionally, the server is also used for:

[0071] Check if there is any stored remote assistance video or operation guide data that matches the target viewing request;

[0072] If it exists, the server sends a matching historical video recording or historical operation guidance request to the first mobile terminal based on the target viewing request, so that the first mobile terminal user can view it;

[0073] If it does not exist, a request failure message is sent to the first mobile terminal, and a fifth query message is output to ask the first mobile terminal user whether a remote assistance request needs to be triggered.

[0074] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the remote assistance method of the mobile terminal described above.

[0075] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the remote assistance method of the mobile terminal described above.

[0076] Using the above technical solution, this application provides a remote assistance method and system for a mobile terminal, a storage medium, and a computer device. In this method, a first mobile terminal responds to a remote assistance trigger operation in a remote assistance application. The first mobile terminal sends a remote assistance request carrying the identifier of a second mobile terminal to a server, accurately establishing a connection with the second mobile terminal. After the connection is established, the remote assistance application in the first mobile terminal activates its screen recording function, records the screen information requiring assistance, and sends it to the second mobile terminal via the server. The second mobile terminal generates an assistance operation instruction based on the recorded screen information and sends it to the first mobile terminal via the server. The first mobile terminal performs the corresponding remote assistance operation according to the assistance operation instruction. This enables the second mobile terminal to remotely operate the first mobile terminal, and the remote assistance operation process can be displayed on the first mobile terminal, providing convenience for the user of the first mobile terminal.

[0077] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0078] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0079] Figure 1 A flowchart illustrating a remote assistance method for a mobile terminal provided in an embodiment of this application is shown.

[0080] Figure 2 A flowchart illustrating another remote assistance method for a mobile terminal provided in an embodiment of this application is shown.

[0081] Figure 3 This paper illustrates a flowchart of another remote assistance method for a mobile terminal provided in an embodiment of this application.

[0082] Figure 4 This illustration shows a structural diagram of a remote assistance system for a mobile terminal provided in an embodiment of this application;

[0083] Figure 5 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0084] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0085] This embodiment provides a remote assistance method for a mobile terminal, such as... Figure 1 As shown, the method includes:

[0086] Step 101: In response to the remote assistance triggering operation in the remote assistance application, the first mobile terminal sends a remote assistance request to the server, wherein the remote assistance request carries the identifier of the second mobile terminal.

[0087] Step 102: The server forwards the remote assistance request to the second mobile terminal, and upon receiving the assistance confirmation feedback sent by the second mobile terminal, forwards the assistance confirmation feedback to the first mobile terminal.

[0088] Step 103: The first mobile terminal starts the screen recording function based on the remote assistance application and sends the recorded screen display data to the second mobile terminal through the server;

[0089] Step 104: The second mobile terminal displays data based on the screen display data and sends an assistance operation command to the server based on the operation data of the display interface.

[0090] Step 105: The server sends the assistance operation instruction to the first mobile terminal, so that the first mobile terminal performs the corresponding remote assistance operation based on the assistance operation instruction.

[0091] This application embodiment can be applied to scenarios of remote assistance between mobile terminals. In this application embodiment, the first mobile terminal is the terminal requesting assistance, and the second mobile terminal is the terminal providing remote assistance. A remote assistance application is installed on both the first and second mobile terminals. For example, the remote assistance application is remote-ctrl, which is an application that includes sending commands, receiving commands, and executing commands. For example, sending commands: On the second mobile terminal, some command information can be sent; receiving commands: The second mobile terminal sends commands to the server, and the server will detect this and forward the command information to the first mobile terminal that needs remote assistance; executing commands: After receiving the commands, the first mobile terminal can call the operating system's functions to execute the received commands; and this remote assistance application has the characteristics of running in the background and starting on boot, and having a small installation package.

[0092] In this embodiment, the first mobile terminal and the second mobile terminal can be mobile terminals such as mobile phones, tablets, and computers. In response to a remote assistance trigger operation in a remote assistance application, the first mobile terminal sends a remote assistance request carrying the identifier of the second mobile terminal to the server. The service request carries the identifier of the second mobile terminal, allowing the server to accurately send the request to the second mobile terminal based on the identifier. This enables the first mobile terminal to select the second mobile terminal to send the assistance request, improving the information security of the first mobile terminal user. When the second mobile terminal receives the assistance request and confirms its assistance to the first mobile terminal user, it sends an assistance confirmation feedback to the server. The server then forwards the assistance confirmation feedback to the first mobile terminal, completing the process. A remote assistance connection is established between the first mobile terminal and the second mobile terminal to enable remote assistance operations between the mobile terminals. The first mobile terminal initiates a screen recording function through the remote assistance application to record the screen information that requires assistance. Based on this screen information, the second mobile terminal can visualize the content requiring assistance. The recorded screen display data is then sent to the second mobile terminal via a server. The second mobile terminal displays the data based on the screen display data and generates assistance operation instructions based on the operation data of the displayed interface. These instructions are then sent to the server. The server forwards the assistance operation instructions to the first mobile terminal, which performs the corresponding remote assistance operation based on the instructions.

[0093] By applying the technical solution of this embodiment, the first mobile terminal responds to the remote assistance trigger operation in the remote assistance application, sends a remote assistance request carrying the identifier of the second mobile terminal to the server, and accurately establishes a connection with the second mobile terminal. After the connection between the first mobile terminal and the second mobile terminal is established, the remote assistance application in the first mobile terminal starts the screen recording function of the first terminal, records the screen information that needs assistance, and sends it to the second mobile terminal through the server. The second mobile terminal generates an assistance operation instruction based on the recorded screen information and sends it to the first mobile terminal through the server. The first mobile terminal performs the corresponding remote assistance operation according to the assistance operation instruction, which can realize the remote operation of the first mobile terminal by the second mobile terminal, and the remote assistance operation process can be displayed on the first mobile terminal, providing convenience for the user of the first mobile terminal.

[0094] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another remote assistance method for a mobile terminal is provided. The remote assistance request includes an assistance question, such as... Figure 2 As shown, the method includes:

[0095] Step 201: The first mobile terminal responds to the remote assistance trigger operation in the remote assistance application by sending a remote assistance request to the server. The server queries whether the stored remote assistance recordings contain a target remote assistance recording that matches the assistance question.

[0096] Step 202: If the target remote assistance video is included, the server sends the target remote assistance video to the first mobile terminal, and forwards the remote assistance request to the second mobile terminal when it receives the first continued assistance request from the first mobile terminal; if the target remote assistance video is not included, the server forwards the remote assistance request to the second mobile terminal.

[0097] Step 203: When the server receives the assistance confirmation feedback sent by the second mobile terminal, it forwards the assistance confirmation feedback to the first mobile terminal.

[0098] Step 204: The first mobile terminal starts the screen recording function based on the remote assistance application and sends the recorded screen display data to the second mobile terminal through the server;

[0099] Step 205: The second mobile terminal displays data based on the screen display data and sends an assistance operation command to the server based on the operation data of the display interface.

[0100] Step 206: The server generates a remote assistance recording based on the assistance question and the screen display data, stores the remote assistance recording, and sends the assistance operation instruction to the first mobile terminal so that the first mobile terminal executes the corresponding remote assistance operation based on the assistance operation instruction.

[0101] In this embodiment, taking a rooted Android phone as the first mobile terminal and any assisting phone as the second mobile terminal, rooting unlocks the restrictions of ordinary users and grants super administrator privileges. The purpose of rooting an Android phone is to obtain super administrator privileges. Since every Android phone has an ADB (Android Debug Bridge) debugging tool, only after rooting the Android phone can one execute ADB commands. ADB commands need to be executed using the ADB tool, such as: restarting the phone command: `adb reboot`; turning on Bluetooth command: `adb shell service call bluetooth_manager 6`; simulating mouse operation: `adb shell input mouse tap 100 500`. The 100 and 500 in `adb shell input mouse tap 100 500` mean clicking the top left corner of the screen (100, 500).

[0102] In this embodiment, the Android phone responds to a remote assistance trigger operation in a remote assistance application by sending a remote assistance request to the server. For example, if the assistance question is how to forward a WeChat message, the server queries the stored remote assistance recordings to see if there is a target remote assistance recording that matches how to forward a WeChat message. If the stored remote assistance recordings contain a recording that matches the current assistance question, the server sends the target remote assistance recording to the Android phone. The Android phone user solves the problem of how to forward a WeChat message based on the target remote assistance recording without needing to send another remote assistance request to the phone providing assistance. If the user does not want to view the recording or does not learn how to forward the message after viewing the recording, they can continue to request assistance. When the first request to continue assistance is received from the Android phone, the remote assistance request is forwarded to the phone providing assistance. If the stored remote assistance recordings do not contain a recording matching the current assistance request, the server will directly forward the remote assistance request. Upon receiving the assistance confirmation feedback, the server will forward the feedback to the Android phone. The Android phone will then initiate screen recording based on the remote assistance application and send the recorded message to be forwarded, along with the screen display data of the person to whom the message is to be forwarded, to the phone providing assistance via the server. The phone providing assistance will then display the received screen display data and, based on the operation data regarding how to interact with the displayed interface, will send assistance operation instructions to the server. The server will generate and store the remote assistance recording based on the method for forwarding the WeChat message and the screen display data, and will then send the assistance operation instructions to the Android phone so that the Android phone can execute the instructions to forward the WeChat message.

[0103] It should be noted that in step 202, "If included, the server sends the target remote assistance video to the first mobile terminal," the first mobile terminal receives the target remote assistance video and outputs a first query message, asking the first mobile terminal user whether to view a historical video matching the currently requested assistance problem. If the first mobile terminal user confirms viewing, the first mobile terminal receives a signal confirming the viewing of historical videos, then plays the target remote assistance video. After playback, it outputs a second query message, asking the first mobile terminal user whether the currently requested assistance problem has been resolved. If the first mobile terminal user confirms that it has not been resolved, the first mobile terminal sends a first continued assistance request to the server upon receiving a signal indicating that the assistance problem has not been resolved. If it receives a signal indicating that the first mobile terminal user refuses to view historical videos, it sends a first continued assistance request to the server.

[0104] By applying the technical solution of this embodiment, when the first mobile terminal user encounters an assistance problem, he / she sends a remote assistance request to the server. If the server contains a remote assistance video recording of the assistance problem, he / she can directly view the stored remote assistance video recording without sending a remote assistance request to the second mobile terminal. If the problem is not resolved, then the assistance request is sent to the second mobile terminal, which provides convenience for the first mobile terminal user.

[0105] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another remote assistance method for a mobile terminal is provided. The remote assistance request includes an assistance question, such as... Figure 3 As shown, the method includes:

[0106] Step 301: The first mobile terminal responds to the remote assistance trigger operation in the remote assistance application and sends a remote assistance request to the server. The server queries whether the stored operation guidance data contains target operation guidance data that matches the assistance question.

[0107] Step 302: If the target operation guidance data is included, the server sends the target operation guidance data to the first mobile terminal, and forwards the remote assistance request to the second mobile terminal when it receives the second continued assistance request from the first mobile terminal; if the target operation guidance data is not included, the server forwards the remote assistance request to the second mobile terminal.

[0108] Step 303: When the server receives the assistance confirmation feedback sent by the second mobile terminal, it forwards the assistance confirmation feedback to the first mobile terminal.

[0109] Step 304: The first mobile terminal starts the screen recording function based on the remote assistance application and sends the recorded screen display data to the second mobile terminal through the server;

[0110] Step 305: The second mobile terminal displays data based on the screen display data and sends an assistance operation command to the server based on the operation data of the display interface.

[0111] Step 306: The server extracts the operation frame image corresponding to the assistance operation instruction from the screen display data, and determines the operation component corresponding to the operation frame image and the operation method of the operation component based on the operation position and operation method corresponding to the assistance operation instruction.

[0112] Step 307: Mark the operation components and operation methods in the operation frame image to obtain an operation guidance image, and store the operation guidance data for the assistance problem;

[0113] Step 308: Send the assistance operation instruction to the first mobile terminal so that the first mobile terminal can perform the corresponding remote assistance operation based on the assistance operation instruction.

[0114] In this embodiment, the first mobile terminal, in response to a remote assistance trigger operation in a remote assistance application, sends a remote assistance request to the server. The server queries the stored operation guidance data to see if it contains target operation guidance data matching the assistance question. If it does, the server sends the target operation guidance data to the first mobile terminal. Upon receiving a second continued assistance request from the first mobile terminal, the server forwards the remote assistance request to the second mobile terminal. If the target operation guidance data is not found, the server forwards the remote assistance request to the second mobile terminal. Upon receiving assistance confirmation feedback, the server forwards the assistance confirmation feedback to the first mobile terminal. The first mobile terminal initiates a screen recording function based on the remote assistance application and sends the recorded screen display data to the second mobile terminal via the server. The second mobile terminal displays the screen based on the screen display data and, based on the operation data of the display interface, sends... The server sends an assistance operation command; the server extracts the operation frame image corresponding to the assistance operation command from the screen display data. The operation frame image is an image of the operation generated by the second mobile terminal extracted from the video recording. Based on the operation position and operation method corresponding to the assistance operation command (operation methods include clicking, dragging, double-clicking, etc.), the server determines the operation component corresponding to the operation frame image and the operation method of the corresponding operation component; the server marks the operation component and operation method in the operation frame image to obtain an operation guidance image. For example, the operation guidance image circles the component clicked by the second mobile terminal and marks how the operation was performed, indicating whether the circled component was clicked or dragged; the server stores the operation guidance data for the assistance problem; the server sends the operation position and operation method corresponding to the assistance operation command to the first mobile terminal so that the first mobile terminal can perform the corresponding remote assistance operation based on the assistance operation command.

[0115] By applying the technical solution of this embodiment, when the first mobile terminal user encounters an assistance problem, a remote assistance request is sent to the server. If the server contains operation guidance data for the assistance problem, the stored operation guidance data can be viewed directly without sending a remote assistance request to the second mobile terminal. If the problem is not resolved, then an assistance request is sent to the second mobile terminal, which provides convenience for the first mobile terminal user.

[0116] It should be noted that the above method also includes: the first mobile terminal sending a target viewing request to the server to query video recordings or operation guidance data; the server querying whether there is a remote assistance video recording or operation guidance data in the stored data that matches the target viewing request; if so, the server sending a matching historical video recording or historical operation guidance request to the first mobile terminal so that the first mobile terminal user can view it; if not, the server sending a request failure message to the first mobile terminal and outputting a fifth query message to ask the first mobile terminal user whether they need to trigger a remote assistance request. This allows the first mobile user to view the stored remote assistance video recordings or operation guidance data at any time without sending a remote assistance request.

[0117] Furthermore, as Figure 1 In terms of specific implementation, this application provides a remote assistance system for a mobile terminal, such as... Figure 4 As shown, the system includes:

[0118] First mobile terminal, second mobile terminal, and server;

[0119] The first mobile terminal is configured to send a remote assistance request to the server in response to a remote assistance trigger operation in a remote assistance application, wherein the remote assistance request carries a second mobile terminal identifier.

[0120] The server is configured to forward the remote assistance request to the second mobile terminal, and upon receiving the assistance confirmation feedback sent by the second mobile terminal, forward the assistance confirmation feedback to the first mobile terminal.

[0121] The first mobile terminal is also configured to initiate a screen recording function based on the remote assistance application and send the recorded screen display data to the second mobile terminal through the server;

[0122] The second mobile terminal is used to display data based on the screen display data and to send assistance operation instructions to the server based on operation data of the display interface.

[0123] The server is further configured to send the assistance operation instruction to the first mobile terminal, so that the first mobile terminal performs the corresponding remote assistance operation based on the assistance operation instruction.

[0124] Optionally, the server is also used for:

[0125] A remote assistance recording is generated based on the assistance question and the screen display data, and the remote assistance recording is stored.

[0126] Optionally, the server is also used for:

[0127] Check if the stored remote assistance recordings contain a target remote assistance recording that matches the assistance question;

[0128] If included, the target remote assistance video is sent to the first mobile terminal, and upon receiving the first continued assistance request from the first mobile terminal, the remote assistance request is forwarded to the second mobile terminal;

[0129] If not included, the remote assistance request is forwarded to the second mobile terminal.

[0130] Optionally, the first mobile terminal is further configured to:

[0131] Receive the target remote assistance video and output a first query message to ask the first mobile terminal user whether to view the historical video that matches the currently requested assistance question;

[0132] If a signal confirming the viewing of historical recordings is received, the target remote assistance recording is played. After the playback ends, a second query message is output to ask the first mobile terminal user whether the current requested assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, the first request to continue assistance is sent to the server.

[0133] If a signal is received indicating that viewing historical recordings is refused, the first request for continued assistance is sent to the server.

[0134] Optionally, the server is also used for:

[0135] Based on the screen display data and the assistance operation instructions, operation guidance data corresponding to the assistance question is generated and stored.

[0136] Optionally, the server is also used for:

[0137] Check whether the stored operation guidance data contains target operation guidance data that matches the assistance question;

[0138] If included, the target operation guidance data is sent to the first mobile terminal, and upon receiving the second continued assistance request from the first mobile terminal, the remote assistance request is forwarded to the second mobile terminal.

[0139] If not included, the remote assistance request is forwarded to the second mobile terminal.

[0140] Optionally, the server is also used for:

[0141] Extract the operation frame image corresponding to the assistance operation instruction from the screen display data, and determine the operation component corresponding to the operation frame image and the operation method of the operation component based on the operation position and operation method corresponding to the assistance operation instruction.

[0142] The operation components and operation methods are labeled in the operation frame image to obtain an operation guidance image, wherein the operation guidance data includes the operation guidance image.

[0143] Optionally, the first mobile terminal is further configured to:

[0144] Receive the target operation guidance data and output a third query message to ask the first mobile terminal user whether to view the historical operation guidance data that matches the currently requested assistance question;

[0145] If a signal confirming the viewing of historical operation guidance data is received, the target operation guidance data is played. After the playback ends, a fourth query message is output to ask the first mobile terminal user whether the current assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, a second request for continued assistance is sent to the server.

[0146] If a signal is received indicating that viewing historical operation guidance data is refused, a second request for continued assistance is sent to the server.

[0147] Optionally, the first mobile terminal is further configured to:

[0148] Send a target viewing request to the server to query video recordings or operation guidance data;

[0149] Optionally, the server is also used for:

[0150] Check if there is any stored remote assistance video or operation guide data that matches the target viewing request;

[0151] If it exists, the server sends a matching historical video recording or historical operation guidance request to the first mobile terminal based on the target viewing request, so that the first mobile terminal user can view it;

[0152] If it does not exist, a request failure message is sent to the first mobile terminal, and a fifth query message is output to ask the first mobile terminal user whether a remote assistance request needs to be triggered.

[0153] It should be noted that other corresponding descriptions of the functional units involved in the remote assistance system for a mobile terminal provided in this application embodiment can be found by referring to... Figures 1 to 3 The corresponding descriptions in the method will not be repeated here.

[0154] This application also provides a computer device, which may specifically be a personal computer, a server, a network device, etc. Figure 5 As shown, the computer device includes a bus, a processor, memory, and a communication interface, and may also include an input / output interface and a display device. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores location information. The network interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0155] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0156] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A remote assistance method for a mobile terminal, characterized in that, The method includes: In response to a remote assistance triggering operation in a remote assistance application, the terminal requesting assistance sends a remote assistance request to the server, wherein the remote assistance request carries an identifier of the terminal providing remote assistance. The server forwards the remote assistance request to the terminal providing remote assistance, and upon receiving the assistance confirmation feedback sent by the terminal providing remote assistance, forwards the assistance confirmation feedback to the terminal requesting assistance. In response to the assistance confirmation feedback, the requesting terminal starts the screen recording function based on the remote assistance application and sends the recorded screen display data to the terminal providing remote assistance through the server. The terminal providing remote assistance displays data based on the screen display data and sends assistance operation instructions to the server based on the operation data of the display interface. The server extracts the operation frame image corresponding to the assistance operation instruction from the screen display data based on the assistance operation instruction, and marks the operation components and operation methods in the operation frame image based on the operation position and operation method corresponding to the assistance operation instruction to obtain an operation guidance image. The operation guidance image includes circling the components that the terminal providing remote assistance can click in the image and marking how to operate them, and marking whether the circled components are clicked or dragged. The server sends the operation guidance image to the terminal requesting assistance, so that the user corresponding to the terminal requesting assistance can perform an operation on the terminal requesting assistance based on the operation guidance image in the remote assistance application.

2. The method according to claim 1, characterized in that, The remote assistance request includes an assistance question; the requesting terminal sends recorded screen display data to the terminal providing remote assistance via the server, including: The terminal requesting assistance sends the recorded screen display data to the server; The server generates a remote assistance recording based on the assistance question and the screen display data, and stores the remote assistance recording. The server queries whether the stored remote assistance recordings contain a target remote assistance recording that matches the assistance question; If included, the target remote assistance video is sent to the terminal requesting assistance, and upon receiving the first continued assistance request from the terminal requesting assistance, the first continued assistance request is forwarded to the terminal providing remote assistance. If not included, the screen display data is forwarded to the terminal providing remote assistance.

3. The method according to claim 2, characterized in that, After the server sends the target remote assistance video recording to the terminal requesting assistance, the method further includes: The terminal requesting assistance receives the target remote assistance video recording and outputs a first query message to ask the user corresponding to the terminal requesting assistance whether to view the historical video recording that matches the current request for assistance. If a signal confirming the viewing of historical recordings is received, the target remote assistance recording is played. After the playback ends, a second query message is output to ask the requesting terminal user whether the current assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, the first request to continue assistance is sent to the server. If a signal is received indicating that viewing historical recordings is refused, the first request for continued assistance is sent to the server.

4. The method according to claim 1, characterized in that, The remote assistance request includes an assistance question; after the terminal providing remote assistance displays data based on the screen display data and sends an assistance operation instruction to the server based on the operation data of the display interface, the method further includes: The server generates operation guidance data corresponding to the assistance question based on the screen display data and the assistance operation instructions, and stores the operation guidance data of the assistance question; The server queries whether the stored operation guidance data contains target operation guidance data that matches the assistance question. If included, the target operation guidance data is sent to the terminal requesting assistance, and upon receiving the second continued assistance request from the terminal requesting assistance, the second continued assistance request is forwarded to the terminal providing remote assistance. If not included, the server extracts the operation frame image corresponding to the assistance operation instruction from the screen display data until the operation guidance image is obtained.

5. The method according to claim 4, characterized in that, The server extracts the operation frame image corresponding to the assistance operation instruction from the screen display data, and marks the operation position and operation method corresponding to the assistance operation instruction in the operation frame image to obtain an operation guidance image, including: The server extracts the operation frame image corresponding to the assistance operation instruction from the screen display data, and determines the operation component corresponding to the operation frame image and the operation method of the operation component based on the operation position and operation method corresponding to the assistance operation instruction. The operation components and operation methods are labeled in the operation frame image to obtain an operation guidance image, wherein the operation guidance data includes the operation guidance image.

6. The method according to claim 4, characterized in that, The method further includes: The terminal requesting assistance receives the target operation guidance data and outputs a third query message to ask the terminal user requesting assistance whether to view historical operation guidance data that matches the current request for assistance. If a signal confirming the viewing of historical operation guidance data is received, the target operation guidance data is played. After the playback ends, a fourth query message is output to ask the terminal user who requested assistance whether the current assistance problem has been resolved. If a signal indicating that the assistance problem has not been resolved is received, a second request for continued assistance is sent to the server. If a signal is received indicating that viewing historical operation guidance data is refused, a second request for continued assistance is sent to the server.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: The terminal requesting assistance sends a target viewing request to the server to query video recordings or operation guidance data. The server queries the stored remote assistance video or operation guide data to see if there is any remote assistance video or operation guide data that matches the target's viewing request. If it exists, the server sends a matching historical video recording or historical operation guidance request to the terminal requesting assistance based on the target viewing request, so that the terminal user requesting assistance can view it. If not, a request failure message is sent to the terminal requesting assistance, and a fifth query message is output to ask the terminal user requesting assistance whether they need to trigger a remote assistance request.

8. A remote assistance system for a mobile terminal, characterized in that, The system includes: The terminal requesting assistance, the terminal providing remote assistance, and the server; The terminal requesting assistance is used to send a remote assistance request to the server in response to a remote assistance triggering operation in a remote assistance application, wherein the remote assistance request carries an identifier of the terminal providing remote assistance. The server is configured to forward the remote assistance request to the terminal providing remote assistance, and upon receiving an assistance confirmation feedback from the terminal providing remote assistance, forward the assistance confirmation feedback to the terminal requesting assistance. The terminal requesting assistance is also configured to, in response to assistance confirmation feedback, initiate screen recording based on the remote assistance application, and send the recorded screen display data to the terminal providing remote assistance through the server; The terminal providing remote assistance is used to display data based on the screen display data and to send assistance operation instructions to the server based on operation data of the display interface. The server is further configured to extract the operation frame image corresponding to the assistance operation instruction from the screen display data, and based on the operation position and operation method corresponding to the assistance operation instruction, mark the operation components and operation methods in the operation frame image to obtain an operation guidance image. The operation guidance image includes a component that the terminal providing remote assistance can click in the image, and annotates how to operate it. It also indicates whether the circled component is clicked or dragged. The operation guidance image is sent to the terminal requesting assistance so that the user corresponding to the terminal requesting assistance can perform the operation on the terminal requesting assistance based on the operation guidance image in the remote assistance application.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images