Electronic device, game assisting method thereof and training method for artificial intelligence module

Abstract

An electronic device, a game assisting method thereof and a training method for an artificial intelligence module are provided. The game assisting method of the electronic device includes the following steps. One of a central processing unit, an integrated graphics processor and a discrete graphics processor is selected as a designated computing unit. A screenshot of the electronic device is captured. A designated computing unit performs a first recognition process on the screenshot using a first recognition model of an artificial intelligence module to recognize whether the screenshot contains a game frame and whether the screenshot has a game prompt. If the screenshot has the game prompt, the designated computing unit performs a second recognition process on the screenshot using a second recognition model of the artificial intelligence module to recognize a display location of the game prompt and an input action.

Description

[Technical Field]

[0001] This disclosure relates to an electronic device and its operation method, and more particularly to an electronic device, its game-aiding method and training method for an artificial intelligence module. [Previous Technology]

[0002] In recent years, online games have become increasingly popular, and the difficulty of game content has become increasingly complex. In particular, some secret techniques and tips are hidden in the game. Users must use these secret techniques or tips in the process of playing the game in order to solve the level.

[0003] However, as the game becomes more difficult, users who are not familiar with the game's tricks and tips will not be able to play smoothly. Therefore, how to enable users to quickly use these tricks and tips is the research direction of this invention. [Summary of the Invention]

[0004] This disclosure relates to an electronic device, its game assistance method, and a training method for an artificial intelligence module, which utilizes pre-collected game prompts to train artificial intelligence. When a user is playing an online game, whenever there is a corresponding game prompt, it can automatically identify and provide prompts or execute them automatically, thereby enhancing the user's gaming experience.

[0005] According to one aspect of this disclosure, a game-assisting method for an electronic device is proposed. The game-assisting method for an electronic device includes the following steps: Based on the load of a Central Processing Unit (CPU), an Integrated Graphics Processing Unit (iGPU), and a Discrete Graphics Processing Unit (dGPU), one of the CPU, iGPU, and dGPU is selected as a designated computing unit. A screenshot of the screen of the electronic device is captured. The designated computing unit performs a first recognition procedure on the screenshot using a first recognition model of an artificial intelligence module to identify whether the screenshot has a game screen and whether the screenshot has a game prompt. If the screenshot has a game screen and a game prompt, the designated computing unit performs a second recognition procedure on the screenshot using a second recognition model of an artificial intelligence module to identify the display position of the game prompt in the screenshot and a corresponding input action.

[0006] According to another aspect of this disclosure, an electronic device is provided. The electronic device includes a central processing unit (CPU), an integrated graphics processing unit (iGPU), a discrete graphics processing unit (dGPU), a control unit, a display screen, a screen capture unit, and an artificial intelligence module. The control unit is used to select one of the CPU, iGPU, and dGPU as a designated computing unit based on the load of the CPU, iGPU, and dGPU. The screen capture unit is connected to the display screen. The screen capture unit is used to capture a screenshot of the electronic device. The artificial intelligence module includes a first recognition model and a second recognition model. The first recognition model specifies that the computing unit performs a first recognition procedure on the screen screenshot using the first recognition model to identify whether the screen screenshot has a game screen and whether the screen screenshot has a game prompt. If the screenshot contains game visuals and game hints, the specified processing unit performs a second recognition procedure on the screenshot using a second recognition model to identify the display position of the game hints in the screenshot and the corresponding input action.

[0007] According to another aspect of this disclosure, a training method for an artificial intelligence module is proposed. The training method for the artificial intelligence module includes the following steps: Providing several game screens. Each game screen has a game prompt. Obtaining a dataset based on each game screen. Each dataset includes resolution information, a display position of the game prompt, an input action of the game prompt, and a lumen output of the game prompt. Training a first recognition model using these game screens and these datasets. The first recognition model is used to identify whether a screenshot contains a game screen and whether the screenshot contains a game prompt. Training a second recognition model using these game screens and these datasets. The second recognition model is used to identify the display position of the game prompt in the screenshot and the corresponding input action.

[0008] In order to better understand the above and other aspects of this disclosure, specific embodiments are described below in conjunction with the accompanying drawings:

Implementation Method

[0009] The technical terms used in this specification are based on common terminology in the field. Where this specification provides explanations or definitions for certain terms, the interpretation of those terms shall be based on the explanations or definitions provided in this specification. Each embodiment disclosed herein has one or more technical features. Where feasible, those skilled in the art may selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in these embodiments.

[0010] Please refer to Figure 1, which illustrates a schematic diagram of a game assistance method of an electronic device 100 according to an embodiment of the present disclosure. When a user plays an online game on the electronic device 100, the game screen GM may display one or more game tips (TPs). For example, the game screen GM in Figure 1 simultaneously displays three different game tips (TPs) to instantly remind the user of the cheat codes that can be used in the game, such as killing enemies, firing continuously, or using special skills.

[0011] In this embodiment, artificial intelligence is trained by pre-collecting relevant game prompts (TPs) from the game. When a user is playing an online game, the system can automatically identify and provide prompts or execute actions automatically when corresponding game prompts (TPs) are available, thereby enhancing the user's gaming experience. The identification procedure is further described below.

[0012] Please refer to Figure 1 and Figure 2 simultaneously. Figure 2 illustrates the first identification procedure PD1 and the second identification procedure PD2. The first identification procedure PD1 is used to identify whether the screenshot FM has a game screen GM and whether the screenshot FM has a game prompt TP. If there is a game prompt TP, then proceed to the second identification procedure PD2.

[0013] Next, as shown in Figures 1 and 2, the second identification program PD2 is used to identify the display position LC of the game prompt TP in the screenshot FM and the corresponding input action AC. The first identification program PD1 can filter out screenshot FMs that do not have a game screen GM or a game prompt TP, thereby speeding up the second identification program PD2. In other words, the two-stage first identification program PD1 and second identification program PD2 can speed up the overall identification efficiency, are suitable for rapidly changing game scenarios, and achieve the effect of real-time identification.

[0014] Please refer to Figure 3, which illustrates an artificial intelligence module 150 according to an embodiment of the present disclosure. The artificial intelligence module 150 includes a first identification model 151 and a second identification model 152. The first identification model 151 is used to execute the first identification procedure PD1 described above, and the second identification model 152 is used to execute the second identification procedure PD2 described above.

[0015] As shown in Figure 3, the first recognition model 151 uses the Convolution 2D layer 1511, the Average Pooling layer 1512 and the activation layer 1513 to recognize whether the screenshot FM has the game screen GM and whether the screenshot FM has the game prompt TP.

[0016] The second identification model 152 is split into two Conv2D layers 1521 and 1522. The output data of Conv2D layer 1521 is input into average pooling layer 1523. The output data of Conv2D layer 1522 and the output data of average pooling layer 1523 are input into max pooling layer 1524.

[0017] The output data of the average pooling layer 1523 and the maximum pooling layer 1524 are merged through the connection layer 1525 and then improved in accuracy through the activation layer 1526 to obtain the display position LC of the game prompt TP in the screen screenshot FM and the corresponding input action AC.

[0018] Please refer to Figures 4 and 5. Figure 4 illustrates a flowchart of a training method for an artificial intelligence module 150 according to an embodiment of this disclosure, and Figure 5 illustrates a dataset DS according to an embodiment of this disclosure. The training method for the artificial intelligence module 150 includes steps S910 to S940. In step S910, as shown in Figure 1, a plurality of game screens (GMs) are provided, each game screen GM having at least one game prompt (TP).

[0019] Next, in step S920, as shown in Figure 5, a data set DS is obtained based on each game screen GM. Each data set DS includes a resolution information RS, a display position LC of the game prompt TP, an input action AC of the game prompt TP, and a lumen LM of the game prompt TP.

[0020] When a screenshot FM contains one game cues TP, the database DS records a set of display position LC, input action AC, and lumen output LM. When a screenshot FM contains two game cues TP, the database DS records two sets of display position LC, input action AC, and lumen output LM, and so on. These two sets of display position LC, input action AC, and lumen output LM are, for example, arranged in the order of the game cues TP from the top left to the bottom right.

[0021] Next, in step S930, as shown in Figures 1 and 4, the first recognition model 151 is trained using these game screens (GM) and the dataset (DS). The first recognition model 151 is used to identify whether the screenshot FM has a game screen (GM) and whether the screenshot FM has a game prompt (TP). In this step, training is only performed on "whether there is a game screen (GM)" and "whether the screenshot FM has a game prompt (TP)," without considering the display position (LC) of the game prompt (PT) on the screenshot FM and the corresponding input action (AC).

[0022] Then, in step S940, as shown in Figures 1 and 4, the second recognition model 152 is trained using these game screens (GM) and these datasets (DS). The second recognition model 152 is used to identify the display position (LC) of the game prompt (PT) on the screen screenshot (FM) and the corresponding input action (AC). In this step, training is only performed on the "display position (LC) of the game prompt (PT) on the screen screenshot (FM) and the corresponding input action (AC)," without considering the content of the screen screenshot (FM).

[0023] By using the training method of the artificial intelligence module 150 described above, a two-stage first recognition model 151 and a second recognition model 152 can be trained. When the game scene changes rapidly, the two-stage first recognition model 151 and second recognition model 152 can minimize latency and quickly recognize the display position LC of the game prompt PT and the corresponding input action AC.

[0024] Please refer to Figure 6, which illustrates a block diagram of an electronic device 100 according to an embodiment of the present disclosure. In this embodiment, the electronic device 100 includes a designated computing unit 110, a control unit 120, a display screen 130, a screen capture unit 140, the aforementioned artificial intelligence module 150, a storage unit 160, an input unit 180, and a prompting unit 190. The display screen 130 is used to display various information, such as a liquid crystal display panel, an OLED display panel, or an electronic paper display panel. The screen capture unit 140 is used to capture images.

[0025] The designated processing unit 110 is selected from one of a central processing unit (CPU) 111, an integrated graphics processing unit (iGPU) 112, and a discrete graphics processing unit (dGPU) 113. The control unit 120 is used to control and manage instruction processing. The artificial intelligence module 150 is used to perform inference and recognition. The input unit 180 is used to input various data and instructions. The prompting unit 190 is used to perform various prompting actions.

[0026] The control unit 120, the image capturing unit 140, the artificial intelligence module 150, the input unit 180, and the prompting unit 190 are, for example, a circuit, a circuit board, a storage device for stored program code, or a chip. The chip is, for example, a programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components or combinations thereof.

[0027] The storage unit 160 is used to record or store data, such as any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD) or similar elements or combinations thereof, and is used to store multiple modules or various applications that can be executed by the processor.

[0028] In this embodiment, when a user is playing an online game, as long as there is a corresponding game prompt PT, it can automatically identify and provide prompts or execute them automatically, thereby enhancing the user's gaming experience. The following flowchart will explain in detail how the above-mentioned components operate.

[0029] Please refer to Figure 7, which illustrates a flowchart of a game-assisted method of an electronic device according to an embodiment of the present disclosure. The game-assisted method of the electronic device 100 includes steps S120, S140, S151, S152, S160, S170, S180, S181, and S190.

[0030] In step S120, as shown in Figure 6, the control unit 120 selects one of the central processing unit 111, integrated graphics processor 112, and discrete graphics processor 113 as the designated computing unit 110 based on their respective loads. For example, the control unit 120 may select the one with the lightest current load as the designated computing unit 110. Alternatively, the control unit 120 may predict the one with the lightest load at a predetermined time in the future as the designated computing unit 110.

[0031] Next, in step S140, as shown in Figure 6, the image capturing unit 140 captures a screen screenshot FM on the display screen 130 of the electronic device 100.

[0032] Then, in step S151, as shown in Figure 6, the designated computing unit 110 uses the first recognition model 151 of the artificial intelligence module 150 to perform a first recognition procedure PD1 on the screenshot FM to identify whether the screenshot FM has a game screen GM and whether the screenshot FM has a game prompt TP. If it is identified that the screenshot FM does not have a game screen or does not have a game prompt TP, then return to step S120; if it is identified that the screenshot FM has a game screen and a game prompt TP, then proceed to step S152.

[0033] Next, in step S152, as shown in Figure 6, the specified computing unit 110 uses the second recognition model 152 of the artificial intelligence module 150 to perform a second recognition procedure PD2 on the screen screenshot FM, so as to identify the display position LC of the game prompt TP on the screen screenshot FM and the corresponding input action AC.

[0034] Then, in step S160, as shown in Figure 6, the control unit 120 determines whether the electronic device 100 is set to an automatic execution mode M1 or a teaching guide mode M2. If the electronic device 100 is set to automatic execution mode M1, then proceed to step S170; if the electronic device 100 is set to teaching guide mode M2, then proceed to step S190.

[0035] In step S170, as shown in Figure 6, the control unit 120 determines whether the screen screenshot FM has multiple game prompts TP. If the screen screenshot FM has multiple game prompts TP, then proceed to step S181; if the screen screenshot FM has only a single game prompt TP, then proceed to step S180.

[0036] In step S180, as shown in Figure 6, the control unit 120 controls the input unit 180 to automatically execute the input action AC.

[0037] In step S181, the control unit 120 controls the input unit 180 to automatically execute the input action AC according to a predetermined sequence. For example, when the screen screenshot FM has multiple game prompts TP, the input unit 180 will execute the input action AC corresponding to the game prompts TP in sequence from the top left to the bottom right.

[0038] In step S190, as shown in Figure 6, the control unit 120 controls the prompting unit 190 to provide a prompt message RM according to the input action AC. The prompt message RM is, for example, a keyboard dynamic lighting effect or a vibration message.

[0039] For example, during the game, when the game prompt TP appears, a specific key on the keyboard will light up to remind the user to press the specific key to use this cheat.

[0040] In the above implementation, when a corresponding game prompt TP appears during the game, the user can automatically execute the game prompt TP or provide a prompt message RM for the game prompt TP according to the user's pre-selection, so that the user can better utilize the game skills and successfully break through the game level, thereby enhancing the game experience.

[0041] In summary, although this disclosure has been presented above with reference to embodiments, it is not intended to limit this disclosure. Those skilled in the art to which this disclosure pertains can make various modifications and refinements without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be determined by the appended claims. [Simplified Explanation of the Diagram]

[0042] Figure 1 illustrates a schematic diagram of a game assistance method for an electronic device according to an embodiment of the present disclosure. Figure 2 illustrates a first identification procedure and a second identification procedure. Figure 3 illustrates an artificial intelligence module according to an embodiment of the present disclosure. Figure 4 illustrates a flowchart of a training method for an artificial intelligence module according to an embodiment of the present disclosure. Figure 5 illustrates a dataset according to an embodiment of the present disclosure. Figure 6 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure. Figure 7 illustrates a flowchart of a game assistance method for an electronic device according to an embodiment of the present disclosure.

Claims

1. A game-assisting method for an electronic device, comprising: Based on the load of a Central Processing Unit (CPU), an Integrated Graphics Processing Unit (iGPU), and a Discrete Graphics Processing Unit (dGPU), one of the CPU, iGPU, and dGPU is selected as a designated computing unit; a screenshot of the screen of the electronic device is captured; the designated computing unit performs a first recognition procedure on the screenshot using a first recognition model of an artificial intelligence module to identify whether the screenshot contains a game screen and whether the screenshot contains a game prompt; if the screenshot contains the game screen and the screenshot contains the game prompt, the designated computing unit performs a second recognition procedure on the screenshot using a second recognition model of the artificial intelligence module to identify the display position of the game prompt in the screenshot and a corresponding input action; and based on the input action, the electronic device automatically executes the input action.

2. The game assistance method for the electronic device as described in claim 1 further includes: Determine whether the electronic device is set to an automatic execution mode or a teaching guide mode; if the electronic device is set to the teaching guide mode, then the electronic device provides a prompt message based on the input action.

3. The game assistance method of the electronic device as described in claim 2, wherein the prompt message is a dynamic keyboard lighting effect.

4. The game assistance method of the electronic device as described in claim 2, wherein the prompt message is a vibration message.

5. The game assistance method for the electronic device as described in claim 1 further includes: Determine whether the electronic device is set to an automatic execution mode or a teaching guide mode; if the electronic device is set to the automatic execution mode, then the electronic device automatically executes the input action based on the input action.

6. An electronic device comprising: A central processing unit (CPU); an integrated graphics processing unit (iGPU); a discrete graphics processing unit (dGPU); a control unit for selecting one of the CPU, iGPU, and dGPU as a designated processing unit based on the load of the CPU, iGPU, and dGPU; a display screen; a screen capture unit connected to the display screen for capturing a screenshot of the electronic device. And an artificial intelligence module, comprising: a first recognition model, wherein the designated computing unit performs a first recognition procedure on the screenshot using the first recognition model to identify whether the screenshot has a game screen and whether the screenshot has a game prompt; and a second recognition model, wherein if the screenshot has the game screen and the screenshot has the game prompt, the designated computing unit performs a second recognition procedure on the screenshot using the second recognition model to identify a display position of the game prompt in the screenshot and a corresponding input action; and an input unit for controlling the input unit to automatically execute the input action based on the input action.

7. The electronic device as described in claim 6 further includes: A storage unit for recording whether the electronic device is set to an automatic execution mode or a teaching guide mode; And a prompting unit, if the electronic device is set to the teaching and guiding mode, the control unit controls the prompting unit to provide a prompting message according to the input action.

8. The electronic device as described in claim 7, wherein the prompt message is a dynamic keyboard lighting effect.

9. The electronic device as described in claim 7, wherein the notification message is a vibration message.

10. The electronic device as described in claim 7, further comprising: A storage unit is used to record whether the electronic device is set to an automatic execution mode or a teaching guide mode. If the electronic device is set to the automatic execution mode, the control unit controls the input unit to automatically execute the input action based on the input action.

Images