Mobile terminal software operation guiding method and system

Abstract

The invention discloses a mobile terminal software operation guiding method and system. The method comprises the steps of receiving target to-be-handled service information input by a user, and obtaining a screen image, shot by a camera, of mobile terminal use software; judging an operation intention of a user according to the target to-be-handled service and a screen image of software used by the mobile terminal; and according to the operation intention of the user, obtaining a pre-stored operation guide corresponding to the operation intention, and outputting the operation guide. According to the method, the flexibility, instantaneity and individuation of teaching can be improved.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a method and system for guiding software operation on a mobile terminal. Background Art

[0002] In today's digital age, smartphones and their applications (apps) and mini-programs have become indispensable tools in people's daily lives. However, for older adults and some users unfamiliar with smartphone operation, mastering new apps or mini-programs through self-exploration is often difficult. These users often encounter numerous difficulties when attempting to complete tasks using apps or mini-programs, such as difficulty finding the correct function entry point, lack of understanding of the operational process, or inability to comprehend interface prompts, resulting in inability to successfully complete the task.

[0003] For example, the medical system is gradually promoting the use of electronic medical systems to improve service efficiency and convenience. However, for the elderly and some users unfamiliar with smartphones, it is often difficult to master the operation of hospital electronic medical systems by self-experience. For example, when registering at the hospital, checking test results, and paying bills, these users may feel confused and even unable to complete the medical treatment process due to unfamiliarity with the operation process. This problem is particularly prominent when elderly people navigate the complex electronic medical systems alone without their children.

[0004] Currently, to address the difficulties encountered by such users when operating apps or mini-programs, common solutions include manual and hands-on instruction and video instruction. Manual and hands-on instruction involves direct guidance from others (such as children, volunteers, or professionals) to help users master the operation methods through on-site demonstrations and step-by-step explanations. Video instruction involves recorded operation tutorial videos that users can watch anytime, anywhere to learn the operation process.

[0005] However, manual, hands-on teaching requires a lot of manpower and requires a high level of professional knowledge and patience from the instructor. In addition, this method cannot meet the immediate needs of users when they encounter new problems. Once the instructor is not present, the user may still not be able to solve the problem on their own. Although video teaching has a certain degree of flexibility, its content is usually fixed and cannot be dynamically adjusted according to the user's specific problems. When users encounter problems that are not covered in the video, they still cannot get effective help, resulting in limited learning effects. Therefore, existing teaching methods are insufficient in flexibility, immediacy and personalization, and it is difficult to meet the actual needs of the elderly and some mobile phone users when operating apps or mini-programs. Summary of the Invention

[0006] Based on this, in response to the above technical problems, a mobile terminal software operation guidance method and system are provided to solve the problems of flexibility, immediacy and personalization in existing teaching methods.

[0007] In a first aspect, a method for guiding software operation on a mobile terminal is provided, the method comprising:

[0008] Receive target to-do business information input by the user, and obtain the screen image of the mobile terminal using the software captured by the camera;

[0009] Determining the user's operation intention based on the target pending business and the screen image of the software used by the mobile terminal;

[0010] Obtaining pre-saved operation instructions corresponding to the operation intention according to the user's operation intention;

[0011] The operation guide is output.

[0012] In the above solution, optionally, judging the user's operation intention based on the target to-do business and the screen image of the mobile terminal using the software includes:

[0013] Performing trapezoidal correction processing on the mobile terminal screen image to obtain a pre-processed screen image;

[0014] Recognizing the image content of the screen image; the image content recognition includes color distribution recognition, input box position and shape recognition, and text content recognition;

[0015] The image content recognition result of the screen image is compared with the image contents of multiple standard images corresponding to the target pending business that are pre-saved, and the standard image with the highest similarity is selected to obtain the operation intention pre-associated with the standard image with the highest similarity.

[0016] In the above solution, optionally, before obtaining the screen image of the mobile terminal using the software captured by the camera, the method further includes:

[0017] Obtain the corresponding target software name according to the target pending business;

[0018] Outputting prompt information indicating whether the target software is installed on the user's mobile terminal;

[0019] When receiving the information input by the user that the mobile terminal has not installed the target software, the pre-saved target software installation guide, target software registration guide, and target software login guide are output.

[0020] In the above solution, further optionally, after outputting the prompt information of whether the user's mobile terminal has installed the target software, the method further includes:

[0021] Receiving information input by a user indicating that the mobile terminal has installed the target software;

[0022] Obtain corresponding preset full-process operation instructions based on the target pending business;

[0023] Output the preset full-process operation instructions corresponding to the target pending business in sequence.

[0024] In the above solution, optionally, outputting the operation guide includes: converting the operation guide into text, sound and animation forms for output.

[0025] In a second aspect, a mobile terminal software operation guidance system is provided, the system comprising:

[0026] The prompt information output module of the target pending business is used to receive the target pending business information input by the user and obtain the screen image of the mobile terminal using the software captured by the camera;

[0027] Operation intention determination module: used to determine the user's operation intention based on the target pending business and the screen image of the software used by the mobile terminal;

[0028] Operation guide acquisition module: used for acquiring the pre-saved operation guide corresponding to the operation intention according to the user's operation intention;

[0029] Operation guide output module: used for outputting the operation guide.

[0030] In the above scheme, optionally, the operation intention determination module is specifically used to perform trapezoidal correction processing on the mobile terminal screen image to obtain a preprocessed screen image; identify the image content of the screen image; the image content recognition includes color distribution recognition, input box position and shape recognition, and text content recognition; compare the image content recognition result of the screen image obtained with the image content of multiple standard images corresponding to the target pending business that are pre-saved, select the standard image with the highest similarity, and obtain the operation intention pre-associated with the standard image with the highest similarity.

[0031] In the above scheme, optionally, the system also includes: a multiple operation guide direct output module: used to receive information input by the user that the mobile terminal has installed the target software; and obtain corresponding multiple operation guides according to the target to-be-done business; and output multiple operation guides of the target to-be-done business in sequence.

[0032] In a third aspect, a computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the mobile terminal software operation guidance method described in the first aspect are implemented.

[0033] In a fourth aspect, a computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the mobile terminal software operation guidance method described in the first aspect.

[0034] This application has at least the following beneficial effects:

[0035] This application uses the screen image of the mobile terminal using the software captured by the camera and the prompt information of the user entering the target pending business to determine the user's operation intention and output the corresponding operation guidance. Therefore, this application can directly provide personalized guidance for the next step based on the user's current operation. Thus, it can dynamically adjust specific issues to correctly guide the user to achieve their goals, improving flexibility, immediacy and personalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a mobile terminal software operation guidance method provided by one embodiment of the present application;

[0037] Figure 2 This is an appearance diagram of a device equipped with a mobile terminal software operation guidance system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0039] In one embodiment, Figure 1 As shown, a mobile terminal software operation guidance method is provided, characterized in that the method includes:

[0040] Step S1: receiving target to-do business information input by the user, and obtaining a screen image of the mobile terminal using the software captured by a camera;

[0041] Step S2: judging the user's operation intention based on the target pending business and the screen image of the software used by the mobile terminal;

[0042] Step S3: obtaining a pre-saved operation guide corresponding to the operation intention according to the user's operation intention;

[0043] Step S4: Output the operation instructions.

[0044] In this embodiment, the user can operate at any time if a problem occurs after a certain step, and then follow the above steps to obtain the operation guide for the next step, thereby ensuring improved flexibility.

[0045] In the aforementioned mobile terminal software operation guidance method, the user's operation intention is determined by the screen image of the mobile terminal software captured by the camera and the prompt information of the user entering the target pending business, and the corresponding operation guidance is output. Therefore, this application can directly provide personalized guidance for the next step based on the user's current operation. Thus, specific issues can be dynamically adjusted to correctly guide the user to achieve their goals, improving flexibility, immediacy, and personalization.

[0046] In one embodiment, determining the user's operation intention based on the target to-do service and the screen image of the mobile terminal using the software includes:

[0047] Performing trapezoidal correction processing on the mobile terminal screen image to obtain a pre-processed screen image;

[0048] Recognizing the image content of the screen image; the image content recognition includes color distribution recognition, input box position and shape recognition, and text content recognition;

[0049] The image content recognition result of the screen image is compared with the image contents of multiple standard images corresponding to the target pending business that are pre-saved, and the standard image with the highest similarity is selected to obtain the operation intention pre-associated with the standard image with the highest similarity.

[0050] In this embodiment, when the camera captures the screen of the mobile terminal, the image may be distorted due to an angle tilt, so a trapezoidal correction process is performed to improve subsequent recognition accuracy.

[0051] Among them, the screenshots of each step of the mobile terminal processing of each target pending business will be pre-stored, recorded as standard images, and saved. The operation instructions corresponding to the operation intention of each operation step will be obtained and saved for subsequent output of the operation instructions.

[0052] In one embodiment, before obtaining the screen image of the mobile terminal using the software captured by the camera, the method further includes:

[0053] Receive a target pending business input by a user, and obtain a corresponding target software name according to the target pending business;

[0054] Outputting prompt information indicating whether the target software is installed on the user's mobile terminal;

[0055] When receiving the information input by the user that the mobile terminal has not installed the target software, the pre-saved target software installation guide, target software registration guide, and target software login guide are output.

[0056] In this embodiment, the user may not know which application software to use to perform the target pending task at the time of operation. Therefore, based on the target pending task input by the user, the user is prompted to install, log in, and register the target software. Of course, the method of steps S1-S3 can also be used to provide real-time guidance during the installation, login, and registration process, or all process steps can be directly output.

[0057] In one embodiment, after outputting the prompt information of whether the user's mobile terminal has installed the target software, the method further includes:

[0058] Receiving information input by a user indicating that the mobile terminal has installed the target software;

[0059] Obtaining corresponding multiple operation instructions according to the target pending business;

[0060] Output multiple operation instructions of the target pending business in sequence.

[0061] In this embodiment, when the learning ability of some users is high, the entire operation guide can be directly output without judging based on real-time operation. After the entire operation guide is output, the user can independently learn and judge the next operation guide.

[0062] In one embodiment, outputting the operation guide includes: converting the operation guide into text, sound, and animation forms for output.

[0063] In this embodiment, the operation instructions are converted into text, sound and animation forms for output so that users can understand and learn them more quickly.

[0064] In one embodiment, a mobile terminal software operation guidance system includes:

[0065] The prompt information output module of the target pending business is used to receive the target pending business information input by the user and obtain the screen image of the mobile terminal using the software captured by the camera;

[0066] Operation intention determination module: used to determine the user's operation intention based on the target pending business and the screen image of the software used by the mobile terminal;

[0067] Operation guide acquisition module: used for acquiring the pre-saved operation guide corresponding to the operation intention according to the user's operation intention;

[0068] Operation guide output module: used for outputting the operation guide.

[0069] In one embodiment, the operation intention determination module is specifically used to perform trapezoidal correction processing on the mobile terminal screen image to obtain a preprocessed screen image; identify the image content of the screen image; the image content recognition includes color distribution recognition, input box position and shape recognition, and text content recognition; compare the image content recognition result of the screen image with the image content of multiple standard images corresponding to the target pending business that are pre-saved, select the standard image with the highest similarity, and obtain the operation intention pre-associated with the standard image with the highest similarity.

[0070] In one embodiment, the system also includes: a multiple operation guide direct output module: used to receive information input by the user that the mobile terminal has installed the target software; and obtain corresponding multiple operation guides based on the target to-do business; and output multiple operation guides of the target to-do business in sequence.

[0071] like Figure 2 As shown, this is a device for running a mobile terminal software operation guidance system of the present application. When using it, the user performs the following method:

[0072] 1) At the beginning of the operation, place the phone in the position shown in the figure. The camera on the top of the phone captures the user's screen content and operations in real time. Artificial intelligence is used to determine the user's operation intention. The touch screen on the top of the phone can display operation animations and text instructions, and provide relevant operation prompts through voice, so that the user can get appropriate guidance for each step of the operation until the task is completed or the operation is learned.

[0073] 2) When the upper touch screen detects that a user is standing in front of the device, it will prompt the user to place the phone in the space below through animation, voice, and text. After detecting that the phone is placed in the designated location, it will prompt the user to select the service they need to handle. If the user has not downloaded the App, it will prompt the user how to download it. After downloading and installing, the user will be prompted to register and log in. After logging in, the upper screen will continue to prompt the user to select the service they need to handle. Then, the artificial intelligence will determine the next operation based on the captured operation and the user's intention, and provide animation, text, and voice guidance through the upper screen until the task is completed.

[0074] For the specific definition of a mobile terminal software operation guidance system, please refer to the definition of a mobile terminal software operation guidance method above, which will not be repeated here. The various modules in the above-mentioned mobile terminal software operation guidance system can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0075] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements the above-mentioned mobile terminal software operation guidance method.

[0076] In one embodiment, a computer program product is further provided, including a computer program / instruction. When the computer program / instruction is executed by a processor, the computer program / instruction involves all or part of the process in the above embodiment method.

[0077] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0078] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A mobile terminal software operation guidance method, characterized in that: The method comprises: Receive target to-do business information input by the user, and obtain the screen image of the mobile terminal using the software captured by the camera; Determining the user's operation intention based on the target pending business and the screen image of the software used by the mobile terminal; Obtaining pre-saved operation instructions corresponding to the operation intention according to the user's operation intention; The operation guide is output.

2. The mobile terminal software operation guidance method according to claim 1, characterized in that: Determining the user's operation intention based on the target pending business and the screen image of the mobile terminal using the software includes: Performing trapezoidal correction processing on the mobile terminal screen image to obtain a pre-processed screen image; Recognizing the image content of the screen image; the image content recognition includes color distribution recognition, input box position and shape recognition, and text content recognition; The image content recognition result of the screen image is compared with the image contents of multiple standard images corresponding to the target pending business that are pre-saved, and the standard image with the highest similarity is selected to obtain the operation intention pre-associated with the standard image with the highest similarity.

3. The mobile terminal software operation guidance method according to claim 1, characterized in that: Before obtaining the screen image of the mobile terminal using the software captured by the camera, the method further includes: Obtain the corresponding target software name according to the target pending business; Outputting prompt information indicating whether the target software is installed on the user's mobile terminal; When receiving the information input by the user that the mobile terminal has not installed the target software, the pre-saved target software installation guide, target software registration guide, and target software login guide are output.

4. The mobile terminal software operation guidance method according to claim 3, characterized in that: After outputting the prompt information of whether the user's mobile terminal has installed the target software, the following further comprises: Receiving information input by a user indicating that the mobile terminal has installed the target software; Obtain corresponding preset full-process operation instructions based on the target pending business; Output the preset full-process operation instructions corresponding to the target pending business in sequence.

5. The mobile terminal software operation guidance method according to claim 1, characterized in that: Outputting the operation guide includes: converting the operation guide into text, sound and animation forms for output.

6. A mobile terminal software operation guidance system, characterized in that: The system comprises: The prompt information output module of the target pending business is used to receive the target pending business information input by the user and obtain the screen image of the mobile terminal using the software captured by the camera; Operation intention determination module: used to determine the user's operation intention based on the target pending business and the screen image of the software used by the mobile terminal; Operation guide acquisition module: used for acquiring the pre-saved operation guide corresponding to the operation intention according to the user's operation intention; Operation guide output module: used for outputting the operation guide.

7. The mobile terminal software operation guidance system according to claim 6, characterized in that: The operation intention determination module is specifically used to perform trapezoidal correction processing on the mobile terminal screen image to obtain a pre-processed screen image; identify the image content of the screen image; the image content recognition includes color distribution recognition, input box position and shape recognition, and text content recognition; compare the image content recognition result of the screen image with the image content of multiple standard images corresponding to the target pending business that are pre-saved, select the standard image with the highest similarity, and obtain the operation intention pre-associated with the standard image with the highest similarity.

8. The mobile terminal software operation guidance system according to claim 6, characterized in that: The system also includes: a multiple operation guide direct output module: used to receive information input by the user that the mobile terminal has installed the target software; and obtain corresponding multiple operation guides according to the target to-be-done business; and output the multiple operation guides of the target to-be-done business in sequence.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images