Game assisting method and system based on visual perception

Abstract

The invention provides a game assisting method and system based on visual perception, and belongs to the field of artificial intelligence and game application. The method comprises the steps that firstly, feature parameters of scene categories and parameters of scene element extraction are preset for a selected game, a game knowledge base and an instruction large model are configured for analysis decision and prompt word generation, and then on the basis of an obtained current game screenshot picture, picture scenes are classified, and the scene categories are determined; based on the scene category, extracting element information, used for inquiring a knowledge base and making a decision, of the current game screenshot picture, and then inquiring a preset game knowledge base to obtain corresponding auxiliary information; the scene category, the element information, the auxiliary information and the player state are analyzed, and whether a current player needs to be reminded or not is analyzed; if reminding is necessary, game suggestions or reminding information is generated, and reminding feedback is conducted on the player in an interactive mode. The user experience is improved, and the universality is high.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and game applications, and specifically relates to a game assistance method and system based on visual perception. Background Art

[0002] With the rapid development of computer technology, computer games, as a form of educational or entertainment-oriented computer program, have become increasingly diverse and enriched in content, with increasingly complex plots and realistic graphics. They are playing an increasingly important role in people's lives. In computer games, individuals typically assume the role of different players in different situations, with each player having a different role or level based on their skills. Players need to continuously learn the game content to improve their gaming skills.

[0003] In existing technology, players often learn game techniques from external strategy websites or videos during gameplay. However, this approach presents the following problems: players need to pause the game to consult strategy guides, disrupting their immersion and resulting in poor real-time performance; static strategy guides cannot adapt to the player's current game state (such as character level, equipment, enemy type, etc.), failing to meet personalized requirements; some game scenes require rapid response, making it difficult for players to obtain effective information in a timely manner and unable to adapt to complex operations. Furthermore, game-assisted tools such as cheat scripts often intrude on the game to obtain information and directly replace human control, disrupting game balance and inconsistent with the original intention and basic rules of the game. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, the present invention aims to provide a game assistance method and system based on visual perception, so that players can obtain strategy information in real time without interrupting the game, without intruding into the game, actively operating the game, or destroying the basic rules of the game, while improving the real-time, personalization and operation level of the player's game.

[0005] In order to achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides a game assistance method based on visual perception, the method comprising:

[0007] Step S1: preset the characteristic parameters of the scene category and the parameters for scene element extraction for the selected game, and configure the game knowledge base and the instruction model to perform analysis and decision making and generate a prompt word template for the reminder;

[0008] Step S2: After the game starts, obtain the current game screenshot;

[0009] Step S3, analyzing the current game screenshot based on visual perception, classifying the scene, and determining the scene category;

[0010] Step S4, based on the scene category, extracting the element information of the current game screenshot picture for querying the knowledge base and decision-making;

[0011] Step S5, according to the determined scene category and the extracted element information, querying the pre-set game knowledge base to obtain the corresponding auxiliary information;

[0012] Step S6, analyzing the scene category, element information, auxiliary information and player state to analyze whether the current player needs to be reminded; if necessary, generating game suggestions or reminding information, and entering step S7; if not necessary, the current time = the current time + the length of each frame, and turning to step S2;

[0013] Step S7, the game suggestions or reminding information are displayed through screen text, marks, colors or voice to interactively prompt the player.

[0014] As a preferred embodiment of the present application, when the current game screenshot picture is obtained in step S2, the end-side lightweight processing is performed through the local device to reduce the delay.

[0015] As a preferred embodiment of the present application, when the scene category is determined in step S3, the scene classification is performed through a lightweight image classification algorithm, or the predetermined icon type, state, color, shape visual features on the predetermined position are checked, or the text information of the predetermined area on the screen is recognized through optical character recognition (OCR), and the scene category of the game is determined according to the predefined rules.

[0016] As a preferred embodiment of the present application, when the scene category is determined in step S3, the scene classification is performed through a two-stage method; specifically including:

[0017] In the first stage, the image is preliminarily classified through a coarse classification algorithm of the image with a lower accuracy but a higher recall rate to obtain the candidate scene corresponding to the image, so as to exclude the non-candidate scene and reduce the number of scenes entering the second stage;

[0018] In the second stage, the game scene corresponding to the image is accurately recognized from the candidate scene through a precise detection method.

[0019] As a preferred embodiment of the present application, in step S4, the elements include the player state, the scene state and the game state.

[0020] As a preferred embodiment of the present application, in step S4, when performing element extraction, the elements are extracted according to the preset element configuration according to the scene category; when performing element extraction, the structured game state data is generated by combining the optical character recognition (OCR) reading of the in-game text information, and the icon type, state, color, and shape visual features in the picture; the preset element configuration defines the position, visual features, and / or text content of the elements, and defines the corresponding semantic information for each element; and the generated structured game state data contains the position, visual features, and / or text content of the elements and the corresponding semantic information.

[0021] As a preferred embodiment of the present application, the game knowledge base is preset according to the game scenes involved, and different game settings correspond to different game knowledge bases; and the game knowledge base includes storage of game strategies, character skill analysis, and level pass strategies, and supports keyword search or vector search.

[0022] As a preferred embodiment of the present application, step S6 is implemented by a large language model (LLM) with a real-time reasoning engine, and the large model is instructed to analyze the current game state and generate game reminder content by a preset prompt word template; and the game level and state of the player are adapted when the reminder content is generated, the frequency and level of the reminder content are determined to ensure the pertinence and effectiveness of the reminder.

[0023] As a preferred embodiment of the present application, in step S6, the time length of each frame is selected in the range of 2 frames per second to 0.5 frames per second.

[0024] In a second aspect, the embodiments of the present application also provide a game assistance system based on visual perception, which comprises a predefinition module, a visual perception module, a scene classification module, an element extraction module, an assistance information generation module, a decision module, a reminder module, and a game knowledge base; wherein,

[0025] The predefinition module is used to preset the feature parameters of the selected game scene category, the parameters of the scene element extraction, configure the game knowledge base, and instruct the large model to analyze and decide and generate the prompt word template of the reminder language;

[0026] The visual perception module is used to obtain the current game screenshot picture after the game starts;

[0027] The scene classification module is used to analyze the current game screenshot picture based on visual perception, classify the picture scene, and determine the scene category;

[0028] The element extraction module is used to extract the element information of the current game screenshot picture for querying the knowledge base and decision-making based on the scene category;

[0029] The auxiliary information generation module is configured to query a pre-set game knowledge base according to the determined scene category and the extracted element information, and obtain corresponding auxiliary information.

[0030] The decision module analyzes the scene category, the element information, the auxiliary information and the player state, analyzes whether the current player needs to be reminded, generates a game suggestion or a reminding information if necessary, and starts the reminding module, and returns to the visual perception module if the reminding is not necessary.

[0031] The reminding module is configured to display the game suggestion or the reminding information through screen display text, marks, colors or voice in an interactive manner to prompt feedback to the player.

[0032] The technical scheme provided by the embodiment of the application has the following beneficial effects:

[0033] The game auxiliary method and system based on visual perception provided by the embodiment of the application extracts state information by analyzing a game picture in real time, generates personalized game strategy suggestions, operation reminders and teaching guidance in combination with a large language model, a game knowledge base and player behavior, and reminds the player in the form of voice / UI. The system adopts a cloud-end collaborative architecture, reduces the computing load while ensuring real-time performance, enables the player to obtain strategy reminders and other reminding information in real time without interrupting the game, obtains targeted and personalized strategy information in combination with the player's own situation and current state, is suitable for different types of games, has universality, conforms to the basic rules of the game, does not intrude into the game or actively operate the game, and improves the operation level and game experience of the player.

[0034] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0036] Figure 1 is a flowchart of the game auxiliary method based on visual perception described in the embodiments of the application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0038] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In the description of the present application, the terms "first", "second", "third", "fourth" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0039] The embodiments of the present application provide a game assistance method and system based on visual perception, which extracts state information by real-time analysis of game pictures, combines large language models and game knowledge bases and player behaviors to generate personalized game strategy suggestions, operation reminders and teaching guidance, and reminds players in the form of voice / UI. The system adopts a cloud-end collaborative architecture to reduce the computing load while ensuring real-time performance, so that players can obtain strategy reminders and other information in real time without interrupting the game, and obtain targeted and personalized strategy information in combination with their own situation and current state. The system is suitable for different types of games, has universality, meets the basic rules of the game, does not invade the game, and does not actively operate the game, and improves the operation level and game experience of players.

[0040] As shown in the game assistance method based on visual perception, the method comprises the following steps: Figure 1

[0041] Step S1, presetting the feature parameters of the selected game scene category, the parameters of the extracted scene elements, configuring the game knowledge base, and instructing the large model to analyze and decide and generate the prompt word templates of the reminder sentences;

[0042] Step S2, after the game starts, obtaining the current game screenshot picture.

[0043] In this step, a frame of game picture is obtained by taking a screenshot of the current game picture. In operation, the screenshot picture is obtained at a predetermined time interval, and each frame of game picture is processed one by one. Here, the processing of the current frame is described. When obtaining the current game screenshot picture, the end-side lightweight processing can be performed through the local device to reduce the delay.

[0044] Step S3, based on visual perception, analyzing the current game screenshot picture, classifying the picture scene, and determining the scene category.​

[0045] In this step, the screen scene is classified and the scene category is determined. Two methods can be used.

[0046] Method one: through a lightweight image classification algorithm, or checking the predetermined icon type at a predetermined position, or through OCR to identify the text information of a predetermined area on the screen, the scene category of the game is determined according to the predefined rules.

[0047] Method two: the scene classification is performed through a two-stage method. In the first stage, a coarse classification algorithm of the image is used to preliminarily classify the image at a lower accuracy but a higher recall rate, to obtain the candidate scene corresponding to the image, so as to exclude the non-candidate scene and reduce the number of scenes entering the second stage. In the second stage, a precise detection method (such as detecting the text of a predetermined area or the image of a predetermined area) is used to accurately identify the game scene corresponding to the image from the candidate scene. In this way, the accuracy and recall rate of the scene classification are ensured, and the efficiency is improved. Preferably, when the precise detection is performed, the detection order of each candidate scene is arranged according to the scene switching rule of the game, and the scene corresponding to the previous frame is used as the first candidate scene for detection. If it is not hit, the next most possible scene in the scene switching rule of the game is used as the second candidate scene for detection. In this way, the hit rate of the detection is improved and the classification efficiency is improved.

[0048] Step S4: based on the scene category, the element information of the current game screenshot picture for querying the knowledge base and decision-making is extracted.

[0049] In this step, the elements include player status, scene status, game status such as battle situation of a battle game, management situation of a management game, battle situation of a tower defense game, and other information such as avatar, blood bar, map mark, and various text information, and the specific content may be different for different games. When the elements are extracted, the elements are extracted according to the element configuration preset according to the scene category. When the elements are extracted, the game text information (such as task description, damage value) can be read in combination with OCR (optical character recognition), and the structured game status data is generated. The preset element configuration defines the position, visual feature, and / or text content of the element, and defines the corresponding semantic information for each element; the generated structured game status data contains the position, visual feature, and / or text content of the element and the corresponding semantic information. For different types of games, only the scene category, scene element configuration, and related knowledge base content need to be modified to adapt. The classified element configuration contains the predetermined scene of the predetermined game, and therefore the predefined scene category can distinguish the predetermined scene of the predetermined game (such as the battle scene of the King Glory). To adapt to more games, only the corresponding scene category feature information needs to be configured to complete the classification. In addition, when the elements are extracted, the layout of the page may be adjusted due to the operation of the player, and some positions may move together as a whole. For example, a status panel may be dragged as a whole, which may not be detected using the absolute coordinates of the screen. Therefore, the embodiment provides a relative positioning method of coordinates, such as the position of the text in the panel is fixed, and the position of the panel in the screen can be detected first, and then the text at the specific position in the panel is extracted.

[0050] Step S5: According to the determined scene category and the extracted element information, the pre-set game knowledge base is queried to obtain corresponding auxiliary information.

[0051] In this step, the game knowledge base is pre-set according to the game scene involved. Different games need to set up corresponding game knowledge bases. The preparation of the game knowledge base is prepared by collecting existing information on the network or by hiring professionals to provide related knowledge. The content of the knowledge can help the player to familiarize and master the game and improve the game skills. The game knowledge base (RAG) includes data such as game strategies, character skill analysis, and level pass strategies, and supports keyword search or vector search.

[0052] Step S6: The scene category, element information, auxiliary information, and player status are analyzed to analyze whether the current player needs to be reminded; if necessary, game suggestions or reminder information are generated, and step S7 is entered; if it is not necessary, the current time = current time + each frame length, and step S2 is entered.

[0053] In this step, according to the obtained scene, element information and auxiliary information, the large model is analyzed and decided to generate relevant suggestions or reminder information. The element information includes scene state (such as in battle), player state (level, experience, skill, blood volume) and the like.

[0054] This step is realized by a large language model (LLM) with a real-time inference engine. Through a preset prompt word template, the large model is instructed to analyze the current game state and decide to generate game reminder content, such as enemy weakness analysis ("the current boss has low resistance to fire attribute"), resource management suggestion ("the backpack is full, it is suggested to discard low-level potions"), task path optimization ("priority is given to completing the branch task to obtain key equipment") and the like. The large model adapts to the game level and state of the player when deciding to generate reminder content, determines the frequency of reminders, the level of reminder content and the like, and ensures that the reminders are targeted and useful.

[0055] Preferably, the large language model can perform cloud joint inference. For example, complex decision requests (such as multi-strategy comparison) are sent to a cloud large model for processing.

[0056] In this step, the time length of each frame is generally selected within the range of 2 frames per second to 0.5 frames per second.

[0057] In step S7, the game suggestion or reminder information is displayed through screen text, markers, colors or voice to interactively prompt the player.

[0058] In this step, the game suggestion or reminder information can be pre-set in the form of a prompt word template and can include multi-modal output, which provides a prompt through voice synthesis (TTS) or in-game floating UI, for example, a voice reminder: "be careful to avoid the fan-shaped attack of the boss!"; screen markers: highlight the location of hidden treasure chests; adaptive frequency control: dynamically adjust the prompt frequency according to the player level (high frequency for novice players and only key reminders for advanced players) and the like.

[0059] Based on the same idea, the embodiment of the application also provides a game assistance system based on visual perception, which comprises the following modules: a predefinition module, a visual perception module, a scene classification module, an element extraction module, an auxiliary information generation module, a decision module, a reminder module and a game knowledge base.

[0060] The predefinition module is used to preset the feature parameters of the selected game scene categories, the parameters for extracting scene elements, configure the game knowledge base, and instruct the large model to analyze and decide the prompt word template for generating reminder words.

[0061] The visual perception module is used to obtain the current game screenshot after the game starts.

[0062] The scene classification module is configured to analyze the current game screenshot based on visual perception, classify the scene of the screenshot, and determine the scene category;

[0063] The element extraction module is configured to extract element information of the current game screenshot for querying the knowledge base and decision-making based on the scene category;

[0064] The auxiliary information generation module is configured to query the pre-set game knowledge base according to the determined scene category and the extracted element information, and obtain corresponding auxiliary information;

[0065] The decision module is configured to analyze the scene category, element information, auxiliary information and player state, analyze whether the current player needs to be reminded, generate game suggestions or reminder information if necessary, and start the reminding module; if it is not necessary to remind, the current time = the current time + the time length of each frame, and return to the visual perception module;

[0066] The reminding module is configured to display the game suggestions or reminder information through screen display text, marks, colors or voice to interactively prompt the player.

[0067] The modules in the embodiment are implemented by a processor, and a memory is appropriately added when storage is needed. The processor can be, but is not limited to, a microprocessor (MPU), a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0068] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of them can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, all or part of the computer program instructions generate the processes or functions described in the embodiments of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).

[0069] In addition, it should be noted that the game auxiliary system based on visual perception and the game auxiliary method based on visual perception described in the embodiments are corresponding. The description and limitation of the method are also applicable to the system, which will not be repeated here.

[0070] The present application will be further described in detail through two specific embodiments.

[0071] Embodiment 1: Real-time battle assistance of Wangzehengyao game.

[0072] Predefined module, performing predefined steps, defining the game scene of the king glory, including game loading scene, hall main interface scene, team and matching scene, hero selection scene scene, battle scene, settlement scene of the end of the battle, etc. Define the classification features of each of the above scenes: game loading scene: select the progress bar as the classification detection feature; Hall main interface scene: the top functional area (gold coins, game coins icon) as the classification detection feature; Team / matching scene: with the team avatar frame (circular arrangement, with "invitation" button) as a feature; Hero selection scene: the middle text area at the top of the screen as a detection feature, the text content is the text prompt information of each stage of the selection: battle scene: the right upper corner gear icon, the zoom in, zoom out button of the left small map as a feature; The settlement scene of the end of the battle: the victory / failure mark big character as a feature. Define the elements and extraction rules that need to be extracted in each scene, such as the current game time (text in a specific area on the screen) in the battle scene, the tower state (normal, destroyed, alarm) in the small map (icon shape in a specific position in the small map) and the current player, teammates and enemy heroes' blood volume, position (their visual features are green / blue / red circles and contained avatars, respectively, and each prototype's light color and dark color represent blood volume), red and blue monsters, dragon, tyrant's state and position (position and icon on the small map) and so on. Preconfigure the game knowledge base of the king glory, which contains the introduction information of each hero, skill information, the relationship of restriction and being restricted, the strategy information of hero battle, operation skill information, etc. Predefine the large model to generate the prompt word template, such as "you are a king glory game assistant, can give the player a beneficial, consistent with the current scene of the reminder based on the current player state information, game scene information and given auxiliary information. The reminder is simple and clear, consistent with the oral expression, generally not more than 50 words. If there is nothing to remind the content, return "no". The player state information is as follows: <player>{player status information}< / player> . The game scene information is as follows: <scene>{scene information}< / scene> . The retrieved auxiliary information is as follows: <context>{auxiliary information}< / context>

[0073] The visual perception module captures the current game screenshot picture;

[0074] ​The scene classification module performs classification detection and matching on the current game screenshot, starting from the first candidate scene. It extracts feature content according to the predetermined classification features of the candidate scene and detects whether it matches the candidate scene. When the extracted features match the classification features of the candidate scene, it is confirmed that the game currently belongs to the candidate scene and the match is successful. If not, the next scene is used as the candidate scene for detection and matching. If all candidate scenes are not matched successfully, it is regarded as an unknown scene. The subsequent steps are skipped and the next frame is waited for. For example, if the scene that is successfully matched here is a battle scene, the result of the scene classification is a battle scene. Furthermore, the detection order of each candidate scene can be arranged according to the scene switching rules of the game. For example, the scene corresponding to the previous frame is used as the first candidate scene for detection. If it does not hit, the next most likely scene in the scene switching rules within the game is used as the second candidate scene for detection. This can improve the detection hit rate and improve the classification efficiency. For example, if the scene classification result of the previous frame is a battle scene, the priority detection this time is whether it belongs to the battle scene. If not, the second priority is to detect the settlement scene at the end of the battle.

[0075] The feature extraction module extracts features from the game screenshots based on the scene configuration of the classification results. Taking a battle scene as an example, the features to be extracted include the current game time in the battle scene, the tower status on the minimap, and the health and location information of the current player, teammates, and enemy heroes. Time information is extracted by performing optical character recognition (OCR) on the pre-configured game time display location. A Hough transform is performed on the minimap area to detect circles of a specified size to locate the hero's location. A portrait match is then used to identify the specific hero name. Players, teammates, and enemy heroes are distinguished by color, and each hero's health information is extracted based on the dark and light areas of the circles. The tower's status is determined by detecting the shape of the icon at the tower's location: normal, destroyed, or alarm. The status of the tyrant, dragon, and red and blue monsters is extracted by combining the game time and refresh rules, and by detecting the icons of the tyrant, dragon, and red and blue monsters at specific locations on the minimap. During feature extraction, if some frames are obscured and cannot be extracted, the feature status of the current frame can be predicted using the extraction results of the previous frame. Here, it is assumed that the player hero is Mi Yue, whose health is almost full, and the enemy hero is Sun Wukong, who is located not far to the upper right, and there are no other heroes nearby, and his current health is detected to be low.

[0076] The auxiliary information generation module queries a pre-set game knowledge base according to the extracted elements to obtain corresponding auxiliary information; the index of the game knowledge base supports vector retrieval and keyword retrieval, and for specific hero information, accurate matching is required, keyword retrieval is used by using the hero name, and vector retrieval is used for natural language text content of other elements; the result documents of the query often have many articles, and therefore similarity can be used for sorting, and the top 5 articles are taken as auxiliary information. It is assumed that the auxiliary information includes skill information of the heroes Mi Yue and Sun Wukong, information of the heroes who are suppressed and the heroes who are suppressed, related battle strategy information, etc.

[0077] The decision module merges the auxiliary information and the scene element information according to a pre-set large model prompt word template to generate a prompt word of the large model, and submits the prompt word to the large model for judgment and generates a prompt: "The enemy Sun Wukong is in your upper right, in a blood state, and your Mi Yue has a suppression effect on it. You can quickly attack and use the invincible big move to harvest."

[0078] The reminding module reminds the player through earphone voice broadcast, and marks the possible path of the enemy on the mini map.

[0079] Through the above-mentioned auxiliary method and system, the experience of the game player is improved.

[0080] Embodiment 2: Game resource management of Legend World.

[0081] When the predefinition module performs the predetermined steps, in the predefinition stage, a game scene is added as a backpack opening scene, and the visual features therein are defined as a panel with a title containing the word "backpack", a close button in the upper right corner, and multiple grids below. Then, the categories and quantity identification rules of various props and items are defined, and the game knowledge base is configured, including the related knowledge of game resources and items. A prompt word template of a large model is configured, such as "You are a game assistant of Legend World, and can give the player a beneficial and scene-compliant reminder based on the current player state information, game scene information and given auxiliary information. Your reminder should be simple and clear, consistent with spoken language, and generally no more than 50 words. If there is nothing to remind, return "none". The player state information is as follows: <player>{player status information}< / player> The game scene information is as follows: <scene>{scene information}< / scene> The retrieved auxiliary information is as follows: <context>{auxiliary information}< / context> ”;

[0082] The visual perception module captures the current game screenshot;

[0083] The scene classification module classifies the current game screenshot, and detects that the scene in which the player's backpack is opened at this time;

[0084] The element extraction module extracts scene elements, obtains the type and quantity of items in the backpack, and detects that there are 50 "low-grade ores" but no "high-grade forging materials" in the player's backpack;

[0085] The auxiliary information generation module queries the relevant guide according to the above-mentioned scene elements, and generates a prompt suggestion "suggest selling 20 low-grade ores and going to the XX coordinate to collect rare ore veins" by combining the relevant game guide with the large model.

[0086] The large model in the decision module makes a judgment and generates a prompt through the retrieved auxiliary information.

[0087] The reminding module reminds the user in the game and automatically generates a navigation path marker.

[0088] As can be seen from the above technical solutions, the game auxiliary method and system based on visual perception according to the embodiments of the present application improve the universality of assistance based on a predefined manner, fully combine the player's own situation and current state to obtain targeted and personalized guide information, and meet the basic rules of the game without invading the game or actively operating the game, thereby improving the player's operation level and game experience.

[0089] The above description is only the preferred embodiments of the present application and the explanation of the applied technical principles, and is not intended to limit the scope of the claimed application, but only represents the preferred embodiments of the present application. Those skilled in the art should understand that the scope of the application involved in the present application is not limited to the technical solutions formed by the predetermined combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

Claims

1. A game assistance method based on visual perception, characterized in that: The method comprises: Step S1: preset the characteristic parameters of the scene category and the parameters for scene element extraction for the selected game, and configure the game knowledge base and the instruction model to perform analysis and decision making and generate a prompt word template for the reminder; Step S2: After the game starts, obtain the current game screenshot; Step S3, analyzing the current game screenshot based on visual perception, classifying the scene, and determining the scene category; Step S4: extracting element information of the current game screenshot for querying the knowledge base and making decisions based on the scene category; Step S5: querying a pre-set game knowledge base based on the determined scene category and the extracted element information to obtain corresponding auxiliary information; Step S6: Analyze the scene type, element information, auxiliary information, and player status to determine whether a reminder is needed for the current player. If a reminder is necessary, generate a game suggestion or reminder message and proceed to step S7. If a reminder is not necessary, calculate the current time = the current time + the duration of each frame and proceed to step S2. Step S7: Display the game suggestion or reminder information on the screen in the form of text, mark, color or voice to provide prompt feedback to the player in an interactive manner.

2. The method according to claim 1, characterized in that When obtaining the current game screenshot in step S2, it is processed lightly on the terminal side and run on the local device to reduce latency.

3. The method according to claim 1, characterized in that Step S3 classifies the screen scene. When determining the scene category, a lightweight image classification algorithm is used, or the visual features of the predetermined icon type, status, color, and shape at the predetermined position are checked, or optical character recognition (OCR) is used to identify the text information in the predetermined area on the screen, and the scene category of the game is determined according to predefined rules.

4. The method according to claim 1, wherein Step S3 classifies the scene in the picture. When determining the scene category, a two-stage method is used to classify the scene. Specifically, the method includes: In the first stage, a coarse classification algorithm is used to preliminarily classify images with a lower accuracy but higher recall rate to obtain candidate scenes corresponding to the images, thereby excluding non-candidate scenes and reducing the number of scenes entering the second stage. In the second stage, precise detection methods are used to accurately identify the game scene corresponding to the image from the candidate scenes.

5. The method according to claim 1, wherein In step S4, the elements include player status, scene status, and game status.

6. The method according to claim 1, characterized in that In step S4, when extracting elements, extraction is performed according to the element configuration preset in the scene category; when extracting elements, optical character recognition (OCR) is used to read the text information in the game, and the icon type, status, color, and shape visual features in the screen are combined to generate structured game state data; the preset element configuration defines the position, visual features and / or text content of the elements, and defines corresponding semantic information for each element; the generated structured game state data also includes the position, visual features and / or text content of the elements and the corresponding semantic information.

7. The method according to claim 1, characterized in that The game knowledge base is pre-set according to the game scenarios involved, and different game settings correspond to the game knowledge base; and the game knowledge base includes storage of game strategies, character skill analysis, level clearance strategies, and supports keyword search or vector search.

8. The method according to claim 1, characterized in that Step S6 is implemented through a large language model (LLM) with a real-time inference engine. It uses preset prompt word templates and instruction large models to analyze the current game status and make decisions to generate game reminder content. When deciding to generate reminder content, the player's game level and status are adapted to determine the frequency of reminders and the level of reminder content.

9. The method according to claim 1, characterized in that The duration of each frame in step S6 is selected within the range of 2 frames per second to 0.5 frames per second.

10. A game assistance system based on visual perception, characterized in that: The system includes a pre-defined module, a visual perception module, a scene classification module, an element extraction module, an auxiliary information generation module, a decision module, a reminder module and a game knowledge base; wherein, The predefined module is used to preset the characteristic parameters of the scene category and the parameters of scene element extraction for the selected game, and configure the game knowledge base and the instruction model to perform analysis and decision making and generate prompt word templates for reminder words; The visual perception module is used to obtain the current game screenshot after the game starts; The scene classification module is used to analyze the current game screenshot based on visual perception, classify the scene, and determine the scene category; The element extraction module is used to extract element information of the current game screenshot for querying the knowledge base and making decisions based on the scene category; The auxiliary information generation module is used to query a pre-set game knowledge base based on the determined scene category and the extracted element information to obtain corresponding auxiliary information; The decision module analyzes the scene category, element information, auxiliary information, and player status to determine whether a reminder is needed for the current player. If a reminder is necessary, a game suggestion or reminder message is generated and the reminder module is activated. If a reminder is not necessary, the current time is calculated as the current time + the duration of each frame, and the result is returned to the visual perception module. The reminder module is used to display game suggestions or reminder information through text, marks, colors or voice on the screen, and provide prompt feedback to the player in an interactive manner.

Images