Mobile game trainer using console controller

Abstract

Systems and methods provide application assistance with a mobile device, including running an application on a computer system. A challenging application state of an application is detected. Assistance for the challenging application state may be determined, wherein the assistance includes displaying one or more assistance frames on the mobile device. Information regarding the determined one or more assistance frames is transmitted to the mobile device.

Description

Technical Field

[0001] Aspects of the present disclosure relate to training assistance in applications, and in particular, aspects of the present disclosure relate to using a mobile device to provide training assistance for a video game running on a second device. Background of the Invention

[0003] Video games often require skill and practice to succeed. Many video games also include a competitive mode that pits users against other users. In addition, video games are often graphics and processing power intensive, and therefore they are tied to a computer system that has the graphics and processing power required to run the video game. These computer systems are often immobile and located in the user's home. Therefore, players who want to practice and hone their skills at video games must be at home. While handheld video game consoles exist, they are often underpowered and may run inferior versions of the same game from a computer system. In addition, a user may not want to play the entire game on their handheld console to practice, and these skills may not be applicable between the handheld console and the computer system.

[0004] Mobile devices such as cell phones and media players have become common items that most people carry with them every day. Although applications have been developed that allow for some combination use with computer systems, there are no applications that fully integrate aspects of mobile devices with larger computer systems.

[0005] It is within this context that aspects of the present disclosure are presented. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The teachings of the present disclosure may be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0007] Figure 1 is a diagram depicting an example of an implementation of a mobile game trainer wirelessly coupled to a computer system running an application according to aspects of the present disclosure.

[0008] Figure 2 is a diagram illustrating an example of an implementation of a mobile game trainer coupled to a computer system running an application according to aspects of the present disclosure.

[0009] Figure 3 is a diagram illustrating an example of an implementation of a mobile game trainer on a mobile device coupled to a game controller and a computer system in accordance with aspects of the present disclosure.

[0010] Figure 4 is a diagram depicting an example of a mobile game trainer on a mobile device coupled to a game controller and a remote computer system in accordance with aspects of the present disclosure.

[0011] Figure 5 is a flow chart depicting an example of a method of implementing a mobile game trainer according to aspects of the present disclosure.

[0012] Figure 6 is a flow chart depicting an example of a method for implementing one or more simultaneously displayed auxiliary frames using a mobile game trainer according to aspects of the present disclosure.

[0013] Figure 7 is a flow chart depicting an example of a method for generating assistance using a mobile trainer based on user statistics according to aspects of the present disclosure.

[0014] Figure 8 is a block diagram depicting a system for implementing a mobile trainer according to aspects of the present disclosure. DETAILED DESCRIPTION

[0015] Although the following detailed description contains many specific details for illustrative purposes, it will be appreciated by any person skilled in the art that many variations and modifications to the following details are within the scope of the present invention. Therefore, the exemplary embodiments of the present invention described below are set forth without losing the generality of the claimed invention and without imposing limitations on the claimed invention.

[0016] According to aspects of the present disclosure, a mobile device can be used with an application running on a computer system to create a trainer for a user of the application. When an application (such as a video game) is running on a computer system, a mobile device running a training application (also referred to herein as a mobile game trainer) can be used. The mobile game trainer can provide additional information to the user of the application to enhance the user's experience of the application. For example, but not limited to, when the application is running on the computer system, the mobile application can display keys or key combinations or controller button presses or controller button press combinations for the user, and the mobile game trainer can provide synchronized help information to the user, such as weak spot locations or item locations in the video game application.

[0017] Implementation

[0018] Figure 1An implementation of a mobile game trainer wirelessly coupled to a computer system running an application is illustrated according to aspects of the present disclosure. As shown, a mobile device 103 runs an auxiliary application 106 to provide assistance to a user running an application on a computer system 101. The computer system can be coupled to a display screen 102, which can display frames from an application 107. In addition, as depicted, the computer system 101 can include a wireless connection 108 (e.g., a transceiver and an antenna) and can be communicatively coupled to the mobile device 103, which can also include a similarly configured wireless connection 109. For example, but not limited to, the computer system can be communicatively coupled to the mobile device via a wireless connection (such as a wireless personal area network (WPAN), a wireless local area network (WLAN), a wireless wide area network (WWAN), a wireless cellular network, etc.). As used herein, the term "communicatively coupled" means that at least one of the coupled devices is capable of transmitting a signal containing information that can be interpreted by the other to the other. Such information can include, but is not limited to, commands, data, text, images, graphics, video, or audio. The signal can pass through any number of intermediate devices, and the format of the signal can change, but the content remains unchanged.

[0019] The application 107 may be controlled by a game controller 105. The trainer application on the mobile device 103 may present the auxiliary frame 106 using the display 104 of the mobile device. In addition, the mobile device may be communicatively coupled to the controller 105 through the computer system 101 via a wired or wireless connection. The controller 105 may be used to input control signals (e.g., button presses, joystick positions, etc.) into the auxiliary application. By way of example (but not limitation), the mobile trainer may implement button training, in which the user may be prompted to input certain button presses or button press combinations that correspond to special or secret button presses or button press combinations for the application.

[0020] The application running on the computer system 101 can be, for example, but not limited to, a video game, a video editing program, a word processor, a spreadsheet, or other interactive application. In the illustrated implementation, the application is a video game. The computer system can be, for example, but not limited to, a personal computer (PC), a game console, or a server. The mobile device 103 can be, for example, but not limited to, a mobile phone, a tablet computer, an e-reader, a mobile game console, or other devices including an integrated screen. In the illustrated video game application example, a character is fighting a dragon. One or more auxiliary frames 106 displayed on the mobile display 104 can include, for example, but not limited to, text, graphics, video, animation, or audio, which identifies, for example, a button press or a button press combination, a key press or a key press combination, a highlighted or outlined area of ​​a frame from an application, a video, an image frame, an animated graphics interchange format (GIF) or similar file, a replay of a frame from an application, or a slow replay of a frame from an application.

[0021] exist Figure 1 In the example shown, one or more auxiliary frames include special movement button presses for the application. These auxiliary frames can be synchronized with the state of the application to provide assistance in the challenging state of the application. For example, but not limited to, as shown in the figure, the auxiliary frame displays a dodge button press combination on the display screen in synchronization with the game so that the user can dodge the dragon's attack.

[0022] Figure 2 2 is a diagram illustrating an implementation of a mobile game trainer coupled to a computer system running an application according to aspects of the present disclosure. In the illustrated implementation, a mobile device 203 running a mobile trainer is communicatively coupled to a computer system 201 via a wired connection 208. The wired connection may be, for example, but not limited to, a universal serial bus (USB), Ethernet, a serial interface, a high-definition multimedia interface, or a similar type of connection. Here, one or more auxiliary frames 206 presented using a mobile display 204 of the mobile device 203 include, for example, but not limited to, a frame 207 from an application running on the computer system 201, wherein one or more regions of the frame are highlighted or outlined. The one or more auxiliary frames may be synchronized with the application to display the one or more auxiliary frames 206 simultaneously with the frame 207 of the application displayed on the display screen 202. This may allow a user to look at their mobile device to obtain assistance with the application while using the application and keep the display screen 202 uncluttered. In addition, the one or more auxiliary frames may be saved to the mobile device, allowing the user to view the auxiliary frames (e.g., different outlined or highlighted regions) when they are away from the computer system. Here, the application frame 207 is from a game in which a character fights a dragon with a spear. The user controls the character using a controller 205 coupled to the computer system 201. One or more auxiliary frames 206 displayed on the mobile device show that the frame includes the dragon with the dragon's weak spot highlighted or outlined, and text pointing to the highlighted area and describing the importance of the area is displayed above the frame.

[0023] For example, but not limited to, in order to display a highlighted area of ​​a frame, the computer system may determine that one or more states of the application are challenging application states. The computer system may also select an auxiliary output based on the determination of challenging application states. Figure 2For the implementation shown, the application may include assets that are highlighted or outlined. The computer system 201 may use these assets to create one or more auxiliary frames, displaying the highlighted or outlined areas within the frame. Alternatively, the system may search for keywords or phrases, and determine the locations of the areas of the frames corresponding to the keywords or phrases based on the keywords or phrases, and highlight or outline these areas. For example, but not limited to, a game developer may include keywords or logos corresponding to the weaknesses of monsters in the game in the application code, and the computer system may apply color effects or outlines to the areas of one or more frames that are determined to be display areas corresponding to the keywords or logos. In yet another alternative implementation, the asset may include one or more static frames that display the areas of the highlighted or outlined frames created by the application developer and stored in the memory to be sent to the mobile device. In addition, in some implementations, text and graphic elements (e.g., arrows) may also be generated on the screen, the text may be generated from phrases or words stored in the memory as auxiliary information, the text may be added to the frame based on one or more developer logos or keywords, and similarly, simple images such as arrows may be added to the frame using logos or keywords. For example, but not limitation, if a particular area of ​​a frame is marked as a weak spot, the text "weak spot" and an arrow headed toward the weak spot may be added to the auxiliary frame and displayed on the mobile display along with one or more auxiliary frames.

[0024] Figure 3 An implementation of a mobile game trainer on a mobile device coupled to a game controller and a computer system is depicted according to aspects of the present disclosure. As shown, a mobile device 303 can run a game trainer application and present one or more auxiliary frames 306. The mobile device can also be coupled to a full-size game controller 305, which can be used to control the mobile trainer application and one or more auxiliary frames 306. One or more auxiliary frames 306 can be received from a computer system 301 that is communicatively coupled to the mobile device 303. For example, but not limited to, the console can be wirelessly or physically coupled to the device. In the illustrated implementation, the computer system 301 includes a wireless connection 308 (e.g., an antenna and a transceiver), and the mobile device 303 includes a similar wireless connection 309. The mobile device 303 can use the wireless connections 309, 308 to communicate wirelessly with the computer system over a network 310. Alternatively, the mobile device and the computer system can be physically connected to the network or to each other. The physical connection can be, for example, but not limited to, an Ethernet cable, an RS232 cable, a USB, etc. Additionally, the game controller 305 may be communicatively coupled to the mobile device via, for example, but not limited to, a physical connection (such as a USB cable, PS / 2 cable, etc.) or a wireless connection (such as wireless PAN, near field, infrared, etc.). The clip 312 may couple the mobile device 303 to the game controller 305 to hold the mobile device in a usable position when the user holds the game controller.

[0025] In the implementation shown, the computer system 301 can generate an auxiliary game including one or more auxiliary frames for the mobile device 303. The auxiliary frame can be obtained from the application running on the computer system 301 and converted into an auxiliary game sent to the mobile device. The auxiliary game can be saved to the mobile device 303 or the computer system can run the auxiliary game and send one or more auxiliary frames from the auxiliary game to the mobile device. In some implementations, the auxiliary game can be generated from the challenging application state of the application. For example, but not limited to, the computer system can detect one or more challenging application tasks or states, and can record the application state before the detected challenging application state. Then the application state recorded before the challenging application state can be used to create an auxiliary application. In this implementation, the application can be, for example, but not limited to, a video game, and the challenging application task or state can be a boss battle or the end state of a video game. The application state before the challenging application state and the initialization information from the application can be used to generate an auxiliary game, thereby allowing a small segment of the user to play the game. For example, the auxiliary game can be a challenging segment of a boss battle from an application or a level in a game.

[0026] Figure 4 4 is a diagram depicting a mobile game trainer on a mobile device coupled to a game controller and a remote computer system according to various aspects of the present disclosure. As shown, a mobile device 403 is communicatively coupled to a remote computer system 401 associated with a display 402. The remote computer system here can be, for example, but not limited to, a server that is not near the display or the mobile device. The server can communicate with the display 402 and the mobile device 403 via a network 410. In addition, the game controller 405 can communicate with the remote computer system 401 via the network 410. To facilitate communication, the display screen 402 may include a wireless connection 411, such as including an antenna and an associated transceiver. Similarly, the mobile device 403 may include a similarly configured wireless connection 409, as may the game controller 405. Each of the devices can be communicatively coupled to each other directly via a wired or wireless connection or via a network such as the Internet or a local area network.

[0027] In this implementation, the mobile game trainer displays one or more auxiliary frames 406 in the mobile display, which are replays of the challenging application state that was originally displayed 407 on the display screen 402. The replay of the challenging application state can also include text and / or graphics and / or audio presented on the scene. The text or graphics or audio can include tips or tricks for users to improve their skills. The computer system can generate replay frames from the application state before the challenging application state and the challenging application state. For example, the computer system can re-render frames from the application state and send these frames to the mobile device, and can create a slow motion replay by inserting additional frames or slowing down the frame rate. Developer logos or keywords can be used to generate text in one or more auxiliary frames to insert phrases or words stored in the memory into the frame. The position of the text can be placed in a default position, such as the upper left, upper right, lower left or lower right of the frame. Alternatively, text can be placed based on the pixel values ​​of the frame, for example, text can be placed in a frame area containing a large block with pixel values ​​corresponding to white or black pixels, which ensures that the text is located in a screen area where no important visual information is presented. In some implementations, the computer system 401 can generate audio instead of text or in conjunction with text, such as using text-to-speech. The computer system can present the audio using a speaker incorporated into the display 402 or otherwise associated with the display.

[0028] How to operate

[0029] Figure 5is a flow chart depicting an example of a method for implementing a mobile game trainer according to aspects of the present disclosure. As shown, a computer system may detect a challenging application state at 501. For example, but not limited to, the computer system may detect based on user statistics that the application has reached a premature end state (e.g., a game over screen) or that the user has spent more time in a particular application task or a particular application state or that the particular task or state is challenging. In some implementations, the application developer may mark certain parts of the application as challenging, and the computer system may use these flags to determine which or which application states are challenging. Next, the computer system may determine the type (or types) of auxiliary frames to be generated for the challenging application state at 502. For example, but not limited to, the auxiliary frame type may be determined by a setting input by the user, or alternatively the application may determine the type of auxiliary frame to be generated based on the type of challenging application state. For example, but not limited to, if the challenging application state is the end state of the application, replay may be selected as the type of auxiliary frame to be displayed. Based on the determined type (or types) of auxiliary frames, the computer system may generate one or more auxiliary frames, as indicated at 503. One or more auxiliary frames may be generated by, for example, but not limited to, saving data corresponding to one or more frames from an application running on a computer system and modifying the data to include additional auxiliary information (such as suggested key presses, or key press combinations, or button presses, or button press combinations, or other information provided by the application developer). The computer system may include a library of tips and tricks for the application, which can be searched to provide the user with key or button press combinations and other useful information, such as item locations or game strategies. Other information provided by the developer may include, for example, but not limited to, flags for in-application events, timing for in-application events, or important locations of in-application assets. Alternatively, one or more auxiliary frames may be obtained from predefined auxiliary frames, which may be created by the application developer or may be generated by the computer system based on application parameters (such as application controls). In some implementations, such as video game applications, game developers may provide data usage information that can be used to generate auxiliary information. For example (but not limited to), games that rely on complex button combinations or sequences (e.g., fighting games like Mortal Kombat, Street Fighter, etc.) typically include a "tutorial mode" or action setting list as part of the game. Button mapping, timing, and sequencing data may be extracted from these existing in-game repositories. Finally, one or more auxiliary frames may be sent to the mobile device, where they are presented at 504. In some implementations, the mobile device may modify one or more auxiliary frames, for example, the mobile device may change the resolution or size of one or more auxiliary frames to fit a mobile display.

[0030] Figure 66 is a flowchart depicting another example of a method for implementing one or more synchronously displayed auxiliary frames using a mobile game trainer according to aspects of the present disclosure. In this example, the application is a game. Here, the game reaches a previous end state at 601, such as, but not limited to, a game end screen or a level restart or level portion restart start command. The computer system can then determine the cause of the end state at 602. For example (but not limited to), the computer system can use game data (such as player health or player position, the position of non-player characters, or other player-related or game state information) to determine the cause of the end state. In addition, in some implementations, the end state can be part of a sequence of two or more challenging application states, and in this case, the computer system can determine the state before the challenging application state as the cause of the end state. For example, but not limited to, the computer system can use tags placed by the developer to determine the time between changes in challenging parts, or game data (such as player health, player stamina, player position, etc.). Once the cause of the game end state is determined, the game can be loaded at a point before the game end state at 603. Meanwhile, before or after loading the game state, one or more auxiliary frames may be generated based on the determined end state and other information about the challenging application state at 604. As discussed above, one or more frames of the challenging application state may be obtained from the game and altered to include the auxiliary information, or developer-made frames having auxiliary information corresponding to the challenging application state may be used.

[0031] In some implementations, a playable auxiliary game can be created from a challenging application state before the end state. The game can include base game initialization data, and the challenging application state can be used together with the base initialization data to create a small application that allows a user to play part of the game on a mobile device. The game initialization data and the challenging application state can be converted into a format available to the mobile device. In addition, the assets required to run the challenging application state can be used and can be converted into a format available to the mobile device. For example, but not limited to, the polygon count of the model or the size or vertex count of the texture can be changed to be compatible with the mobile device. The auxiliary application can be run on a computer system and streamed to the mobile device over a network, or alternatively the auxiliary game can be loaded into the memory of the mobile device and run on the mobile device.

[0032] In an alternative implementation, the auxiliary information can be generated by auxiliary logic, which determines which auxiliary information is most relevant based on the basic data collected by the game system during the game before the end of the challenging state. For example, when the player character dies while fighting the Boss AI, the user can reach the end of the challenging state. The game system can record certain information related to the game state at this time, such as which game, which level, which boss, which weapon, etc. The auxiliary logic can use this information to find or generate a hint to defeat the boss. For example, there may be an entire hint library generated by other players for one or more games including this particular game. The auxiliary logic can reveal all hints to the user and let them choose. The logic may include a certain type of artificial intelligence that uses the state end data to predict what assistance to show to the user.

[0033] The auxiliary tips can be open source or created by other users, and the auxiliary generation system can track which tips have been liked by other users. Therefore, if a user finds a tip useful, they can like it, and then determine which tips to reveal based on the game data of the user who provided the tip. For example, many users can provide tips for defeating the boss in the example above. However, if the player character of one of those users has a setup, weapon, and armor combination that matches another player facing the same boss, then this tip may be more relevant to that player trying to defeat the boss.

[0034] After generating one or more auxiliary frames, the one or more auxiliary frames may be presented using a mobile device, as indicated at 605. For example, a computer system may send one or more auxiliary frames to a mobile device, and the mobile device may be configured to present the auxiliary frames sent from the computer system on a display. Such a display may include a screen configured to display text, graphics, images, videos, etc. The display may also include one or more audio speakers and suitable driver hardware and / or software. The auxiliary frames may be stored in a memory of the mobile device before they are displayed. Alternatively, the mobile device may generate one or more auxiliary frames based on information stored in the memory of the mobile device, and display one or more of the generated frames. For example, the mobile device may run an auxiliary game or generate an image frame containing auxiliary information. Finally, the game may be resumed at 606. In some implementations, the auxiliary information may be displayed simultaneously with the resumption of the game, allowing the user to receive assistance for the game on the mobile device while playing the game. In this way, the user can get useful tips while playing the game without taking up screen space.

[0035] Figure 7The flowchart shown depicts an example of a method for generating assistance using a mobile trainer based on user statistics according to various aspects of the present disclosure. In this implementation, user statistics can be determined based on the execution history of the application, as indicated at 701. User statistics can be determined by, for example, but not limited to, monitoring the computer system executing the application for user statistics when the application is running or by receiving user statistics from the application. The application can provide additional information to the computer system to help determine user statistics, for example, but not limited to, the application can provide the computer system with a log of key strokes or button press combinations, frame or scene timing, or game selection. In some implementations, the application can be a game, and the application can provide the computer system with game-related statistics, such as buttons or button press combinations related to its game character, keys or button combinations not used by players, time spent in a specific area, average time spent by most players in a specific area, number of items missed, etc. Based on the determined application statistics, the computer system can determine the type of assistance for the application 702. For example (but not limited to this), if the user is not using or relatively not using a specific key press or button combination, the system can determine an assistance type that displays unused or relatively unused keys or button combinations. In another example, if the application statistics show that the user spends too much time in a particular part of the application, the system can determine an auxiliary type such as a replay or tutorial video. Relatedly, if the user misses a particular item or asset or a portion of an asset in the application; a frame that highlights the particular item or asset or a portion of an asset can be determined to be an auxiliary type. Once the auxiliary type is determined, one or more auxiliary frames are generated by the computer system at 703, or information corresponding to the auxiliary frames is generated and sent to the mobile device, where one or more auxiliary frames are generated. As discussed above, one or more frames can be obtained from the application, or one or more frames can be generated from application statistics or other application information (such as image frames). After the auxiliary frames are generated, the auxiliary frames can be displayed on the mobile device at 704. The mobile device can generate auxiliary frames based on information stored in the mobile device memory, or the auxiliary frames can be sent to the mobile device for storage or display.

[0036] Figure 8 A system configured to implement a mobile trainer is diagrammatically depicted according to aspects of the present disclosure. By way of example (but not limitation), according to aspects of the present disclosure, the mobile trainer system 800 may include a combination of a computer system such as an embedded system, a personal computer, a workstation, a game console, and a mobile device 821 such as a mobile phone, a portable game console, an e-reader, a tablet computer, etc.

[0037] The computer system typically includes a central processing unit (CPU) 803 and memory 804. The computer system may also include well-known support functions 806, which may communicate with other components of the computer system, for example, via a data bus 805. Such support functions may include, but are not limited to, input / output (I / O) elements 807, power supply (P / S) 811, clock (CLK) 812, and cache 813.

[0038] In addition, the mobile device 821 typically includes a CPU 823 and a memory 832. The mobile device 821 may also include well-known support functions 826, which may communicate with other components of the mobile device, for example, via a data bus 825. Such support functions may include, but are not limited to, I / O elements 827, P / S 828, CLK 829, and cache 830. A game controller 835 may optionally be coupled to the mobile device 821 via the I / O elements 827. The game controller 835 may be used to interface with the mobile device 821. The mobile device 821 may also be communicatively coupled to a computer system via the I / O 827 of the mobile system 821 and the I / O elements 807 of the computer system.

[0039] The computer system includes a mass storage device 815, such as a disk drive, a CD-ROM drive, a flash memory, a solid state drive (SSD), a tape drive, etc., to provide non-volatile storage for programs and / or data. The computer system may also optionally include a user interface unit 816 to facilitate interaction between the computer system and the user. The user interface 816 may include a keyboard, a mouse, a joystick, a light pen, or other devices that can be used in conjunction with a graphical user interface (GUI). The computer system may also include a network interface 814 to enable the device to communicate with other devices via a network 820. The network 820 may be, for example, a local area network (LAN), a wide area network (such as the Internet), a personal area network (such as a Bluetooth network), or other types of networks. These components may be implemented in hardware, software, or firmware, or some combination of two or more of these.

[0040] Similarly, the mobile device 821 includes a mass storage device 831, such as a disk drive, a CD-ROM drive, a flash memory, an SSD, a tape drive, etc., to provide non-volatile storage for programs and / or data. The mobile device may also include a display 822 to facilitate interaction between the mobile device or the mobile trainer system and the user. The display may include a screen configured to display text, graphics, images, or videos. In some implementations, the display 822 may be a touch-sensitive display. The display 822 may also include one or more speakers configured to present sounds, such as voice, music, or sound effects. The mobile device 821 may also include a network interface 824 to enable the device to communicate with other devices via a network 820. The network 820 may be, for example, a wireless cellular network, a local area network (LAN), a wide area network (such as the Internet), a personal area network (such as a Bluetooth network), or other types of networks. These components may be implemented in hardware, software, or firmware, or some combination of two or more of these.

[0041] The CPU 803 of the computer system may include one or more processor cores, for example, a single core, two cores, four cores, eight cores, or more cores. In some implementations, the CPU 803 may include a GPU core or multiple cores of the same accelerated processing unit (APU). The memory 804 may be in the form of an integrated circuit that provides addressable memory (e.g., random access memory (RAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), etc.). The main memory 804 may include applications 819 used by the CPU 803 to generate, for example, drawing programs, spreadsheets, video games, word processors, etc. The main memory 804 may also include user statistics 810 that may be generated during the processing of the application 819. The auxiliary information 808 stored in the memory 804 may be used by the CPU 803 to generate one or more auxiliary frames 809. The auxiliary information 808 may include processor execution Figure 6 , Figure 7 and Figure 8 In addition, the auxiliary information 808 may include a library of tips and tricks for the user or a frame of a tutorial video or guide.

[0042] The CPU 823 of the mobile device 821 may include one or more processor cores, for example, a single core, two cores, four cores, eight cores or more cores. In some implementations, the CPU 823 may include a GPU core or multiple cores of the same APU. The memory 832 may be in the form of an integrated circuit that provides an addressable memory (e.g., RAM, DRAM, SDRAM, etc.). The main memory 832 may include an auxiliary frame 834 received from a computer system or generated by the mobile device CPU 823 based on auxiliary information 833 also stored in the main memory 832. The auxiliary frame 834 may be presented with the mobile device display 822. The mass storage 831 of the mobile device 821 may store auxiliary information 833 and auxiliary frames 834 when the CPU 823 is not operating.

[0043] The computer system's mass storage 815 may contain applications or programs 817 that are loaded into the main memory 804 when processing begins on an application 819. In addition, the mass storage 815 may contain data 818 that the processor uses to generate application auxiliary information 808 or auxiliary frames 809.

[0044] As used herein and generally understood by those skilled in the art, an application specific integrated circuit (ASIC) is an integrated circuit that is customized for a specific purpose and is not intended for general use.

[0045] As used herein and generally understood by those skilled in the art, a field programmable gate array (FPGA) is an integrated circuit designed to be configured by a customer or designer after manufacture (hence "field programmable"). FPGA configuration is typically specified using a hardware description language (HDL) similar to that used for ASICs.

[0046] As used herein and generally understood by those skilled in the art, a system on a chip or system on a chip (SoC or SOC) is an integrated circuit (IC) that integrates all the components of a computer or other electronic system into a single chip. It can contain digital, analog, mixed signal and usually radio frequency functions - all on a single chip substrate. Typical applications are in the field of embedded systems.

[0047] A typical SoC includes the following hardware components:

[0048] One or more processor cores (e.g., microcontroller, microprocessor, or digital signal processor (DSP) cores.

[0049] Memory blocks such as read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), and flash memory.

[0050] A timing source, such as an oscillator or a phase-locked loop.

[0051] Peripheral devices such as counter-timers, real-time timers, or power-on reset generators.

[0052] External interfaces, for example, industry standards such as Universal Serial Bus (USB), FireWire, Ethernet, Universal Asynchronous Receiver / Transmitter (USART), Serial Peripheral Interface (SPI) bus.

[0053] Analog interfaces, including analog-to-digital converters (ADCs) and digital-to-analog converters (DACs).

[0054] Voltage regulators and power management circuits.

[0055] These components are connected by a proprietary or industry standard bus. A direct memory access (DMA) controller routes data directly between the external interface and the memory, bypassing the processor core and thereby increasing the data throughput of the SoC.

[0056] A typical SoC includes both the hardware components described above and executable instructions (eg, software or firmware) that control the processor core, peripherals, and interfaces.

[0057] Various aspects of the present disclosure provide enhanced training for the use of computer applications by integrating mobile devices with various aspects of a computer system. This utilizes the computing and graphics resources of the mobile device for training without placing too much of a burden on the resources of the computer system. In addition, the compactness and versatility of the mobile device make it convenient for training and assistance.

[0058] Although the above is a complete description of the preferred embodiment of the present invention, various alternatives, modifications and equivalents may be used. Therefore, the scope of the present invention should not be determined with reference to the above description, but should instead be determined with reference to the appended claims and their full range of equivalents. Any feature described herein (whether preferred or not) may be combined with any other feature described herein (whether preferred or not). In the appended claims, Indefinite article "A" or "An" Refers to one or more quantities of the item following the article unless expressly stated otherwise. The appended claims should not be interpreted as including means-plus-function limitations unless such a limitation is explicitly recited in a given claim using the phrase "means for..."

Claims

1. A method for providing application assistance using a mobile device, comprising: a) running the application on a computer system; b) detecting a challenging application state of the application; c) determining assistance for the challenging application state, wherein the assistance comprises displaying one or more assistance frames on the mobile device; as well as d) sending information about the one or more auxiliary frames to the mobile device. 2 . The method of claim 1 , wherein detecting the challenging application state comprises detecting that the application reaches an end state. 3 . The method of claim 2 , wherein one or more application states prior to the end state are determined to be one or more challenging application states. 4 . The method of claim 3 , wherein d) further comprises restarting the application at a state prior to the one or more challenging application states. 5 . The method of claim 1 , wherein the information regarding the one or more auxiliary frames comprises displaying a slowed replay of image frames from the application.

6. The method of claim 1, wherein the information regarding the one or more auxiliary frames comprises displaying one or more image frames showing user key or button presses on the mobile device.

7. The method of claim 1, wherein the information about the one or more auxiliary frames includes displaying one or more image frames showing secret or lesser used key presses or key press combinations or button presses or button press combinations on the mobile device.

8. The method of claim 1, wherein sending the information about the one or more auxiliary frames comprises displaying one or more frames from the application including a highlighted or outlined area of ​​the frame.

9. The method of claim 8, wherein the application is a game and the area of ​​the frame that is highlighted or outlined is an enemy's weak spot or a missed item or a missed location.

10. The method of claim 1, wherein the detecting the challenging application state comprises analyzing user statistics, and wherein determining the assistance comprises determining unused key presses or key combinations or button presses or button press combinations from the user statistics. The method of claim 1 , wherein determining the assistance comprises searching a library of tips or tricks for the application.

12. The method of claim 11, wherein displaying the one or more auxiliary frames comprises displaying one or more image frames from a video sequence of a tip or trick.

13. The method of claim 1, wherein displaying the one or more auxiliary frames on the mobile device comprises displaying frames from the application modified for use with the mobile device.

14. The method of claim 13, wherein the application is an interactive game and the frames from the application modified for use with the mobile device are part of an interactive application running on the mobile device.

15. A system for providing application assistance, comprising: processor; a memory coupled to the processor; a mobile device coupled to the processor; The non-transitory instructions stored in the memory, when the non-transitory instructions are executed by the processor, cause the processor to perform a method for providing application assistance, comprising: a) running an application using the processor; b) detecting a challenging application state of the application; c) determining assistance for the challenging application state, wherein the assistance comprises displaying one or more assistance frames on the mobile device; and d) sending information about the one or more auxiliary frames to the mobile device.

16. The system of claim 15, wherein detecting the challenging application state comprises detecting that the application reaches an end state. 17 . The system of claim 16 , wherein one or more application states prior to the end state are determined to be one or more challenging application states.

18. The system of claim 17, wherein d) further comprises restarting the application to a state prior to the one or more challenging application states.

19. The system of claim 15, wherein sending the information about the one or more auxiliary frames comprises displaying a slowed replay of image frames from the application.

20. The system of claim 15, wherein transmitting the information about the one or more auxiliary frames comprises displaying one or more image frames showing user key or button presses on the mobile device.

21. The system of claim 15, wherein sending the information about the one or more auxiliary frames comprises displaying one or more image frames showing secret or lesser used key presses or key press combinations or button presses or button press combinations on the mobile device.

22. The system of claim 15, wherein sending the information about the one or more auxiliary frames comprises displaying one or more frames from the application including a highlighted or outlined area of ​​the frame.

23. The system of claim 22, wherein the application is a game and the area of ​​the frame that is highlighted or outlined is an enemy weakness or a missed item or a missed location.

24. The system of claim 15, wherein detecting the challenging application state comprises analyzing user statistics, and wherein determining the assistance comprises determining unused key presses or key combinations or button presses or button press combinations from the user statistics.

25. The system of claim 15, wherein determining the assistance comprises searching a library of tips or tricks for the application.

26. The system of claim 25, wherein displaying the one or more auxiliary frames comprises displaying one or more image frames from a video sequence of a tip or trick.

27. The system of claim 15, wherein displaying the one or more auxiliary frames on the mobile device comprises displaying frames from the application modified for use with the mobile device.

28. The system of claim 27, wherein the application is an interactive game and the frames from the application modified for use with the mobile device are part of an interactive application running on the mobile device.

Images