Program, information processing method, and method for generating a trained model

Abstract

Reduce the effort required for debugging. [Solution] A program that enables a computer device to implement a trial function that attempts to input an operation to a game in a first state using a first trained model; a recording function that, when the operation input from the trial function results in a second state, which is a state in which a predetermined change has occurred in the first state, records the first action set information indicating the content of the operation input and the second image information relating to the second image output in the second state in association with each other; and an acquisition function that acquires second action set information relating to the content of an operation that is presumed to cause a transition from the first state to the second state, based on a plurality of first action set information and a plurality of second image information.

Description

Technical Field

[0001] The present disclosure relates to a program, an information processing method, and a trained model generation method.

Background Art

[0002] Non-Patent Document 1 discloses a technique for operating a game character using AI (Artificial Intelligence), which is a machine learning model. Specifically, Non-Patent Document 1 discloses a technique that uses a language model such as GPT-4o to cause a character to execute an action estimated to be suitable for the current game situation, based on game play logs from a large number of players (including mouse and keyboard operation logs, and game screenshots of the game screen) and a screenshot showing the current game situation.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of Invention

Problem to be Solved by the Invention

[0004] Incidentally, in recent years, AI has been used to perform debugging tasks for games. When having an AI perform gameplay, actions must be defined. However, if an action is simply defined as pressing a single button or key, it becomes difficult to have the AI ​​perform meaningful and complex actions. For example, if it is meaningful in the game for the AI ​​to perform the same operation N (where N is an integer greater than or equal to 1) times in a row (i.e., to bring about some change in the game), the probability of that operation decreases as N increases, making it difficult to have the AI ​​perform such an operation.

[0005] To solve these problems, one possible approach is for humans to define a series of input commands representing meaningful actions within a game as an "action set," and then have the AI ​​perform debugging according to that action set. However, even in this case, debugging would require numerous action sets, and each action set would need to be defined for each game, resulting in significant workload. Reducing the workload involved in creating action sets is also desirable.

[0006] Furthermore, the technology described in Non-Patent Document 1 requires game play logs from a large number of people. In other words, the technology described in Non-Patent Document 1 is based on the premise of test play by a large number of people and does not contribute to reducing the man-hours required during debugging.

[0007] An objective of at least one embodiment of this disclosure is to reduce the effort required for debugging. [Means for solving the problem]

[0008] From a non-limiting perspective, a program according to one embodiment of this disclosure is: In a computer device, A transition function that moves the game state to the first state where the first image is output. A trial function that attempts to input random or predetermined rules of operation to the game in the first state using a first trained model capable of inputting operations to the game, When the state of the game changes to a second state, which is a predetermined change from the first state, as a result of at least one operation input by the trial function, a recording function records a first action set information indicating the content of the at least one operation input and a second image information relating to the second image output in the second state. A repeat function that repeats the series of operations performed by the transition function, the trial function, and the recording function until a predetermined termination condition is met. Based on the plurality of first action set information and plurality of second image information recorded by the recording function, an acquisition function is realized to acquire at least one second action set information relating to an operation that is presumed to transition the game from a first state to a second state. The acquisition function includes a first acquisition function that inputs a plurality of first action set information and a plurality of second image information and a third image information relating to a third image which is an image that can be output in the second state and is different from the plurality of second images into a predetermined language model and acquires from the predetermined language model the second action set information which indicates an operation that is presumed to transition the game from the first state to the second state in which the third image is output.

[0009] From a non-limiting perspective, an information processing method according to one embodiment of the present disclosure is: In a computer device, A transition step that moves the game state to the first state where the first image is output, A trial step in which a first trained model capable of inputting operations to the game is used to attempt random or predetermined rule-based operation inputs to the game in the first state, If, as a result of at least one operation input in the trial step, the state of the game changes to a second state which is a predetermined change from the first state, a recording step is performed to record, in association with, first action set information indicating the content of the operation input and second image information relating to the second image output in the second state. A repeat step which repeats the series of operations in the transition step, the trial step, and the recording step until a predetermined termination condition is met, The process includes: an acquisition step of acquiring at least one second action set of information relating to an operation that is presumed to transition the game from a first state to a second state, based on a plurality of first action set pieces of information and a plurality of second image pieces of information recorded in the recording step; The acquisition step includes inputting a plurality of first action set information and a plurality of second image information and a third image information relating to a third image which is an image that may be output in the second state and is different from the plurality of second images into a predetermined language model, and acquiring second action set information from the predetermined language model that indicates an operation that is presumed to transition the game from the first state to the second state which outputs the third image.

[0010] From a non-limiting perspective, a trained model generation method according to one embodiment of the present disclosure is: In a computer device, A transition step that moves the game state to the first state where the first image is output, A trial step in which a first trained model capable of inputting operations to the game is used to attempt random or predetermined rule-based operation inputs to the game in the first state, If, as a result of at least one operation input in the trial step, the state of the game changes to a second state which is a predetermined change from the first state, a recording step is performed to record, in association with, first action set information indicating the content of the operation input and second image information relating to the second image output in the second state. A repeat step which repeats the series of operations in the transition step, the trial step, and the recording step until a predetermined termination condition is met, a generation step of generating a second trained model trained using learning data in which the plurality of pieces of first action set information recorded in the recording step are used as objective variables and the plurality of pieces of second image information are used as explanatory variables; and causing the above steps to be executed. [Effects of the Invention]

[0011] One or more deficiencies are solved by each embodiment of the present application. [Brief Description of the Drawings]

[0012] [Figure 1] It is a diagram showing an example of the hardware configuration of a computer device corresponding to at least one embodiment of the present disclosure. [Figure 2] It is a diagram showing an example of a functional configuration of a computer device corresponding to at least one embodiment of the present disclosure. [Figure 3] It is a diagram showing an example of a predetermined change in a game corresponding to at least one embodiment of the present disclosure. [Figure 4] It is a flowchart showing an example of an information processing method corresponding to at least one embodiment of the present disclosure. [Figure 5] It is a flowchart showing an example of a first acquisition process corresponding to at least one embodiment of the present disclosure. [Figure 6] It is a diagram showing an example of data input to a language model and data output from the language model in the first acquisition process shown in FIG. 5. [Figure 7] It is a flowchart showing an example of a second acquisition process corresponding to at least one embodiment of the present disclosure. [Figure 8] It is a diagram showing an example of data input to a language model and data output from the language model in the second acquisition process shown in FIG. 7. [Figure 9] It is a diagram showing an example of a functional configuration of a computer device corresponding to at least one embodiment of the present disclosure. [Figure 10]It is a flowchart illustrating an example of an information processing method corresponding to at least one embodiment of the present disclosure. [Figure 11] It is a diagram illustrating an example of a functional configuration of a computer device corresponding to at least one embodiment of the present disclosure. [Figure 12] It is a flowchart illustrating an example of an information processing method (third acquisition process) corresponding to at least one embodiment of the present disclosure. [Figure 13] It is a diagram illustrating an example of data input to the second trained model and data output from the second trained model in the third acquisition process shown in FIG. 12. DETAILED DESCRIPTION OF EMBODIMENTS

[0013] Hereinafter, examples of embodiments of the present disclosure will be described with reference to the drawings. Various constituent elements in the example of each embodiment described below can be appropriately combined within a range where no contradiction or the like occurs. For contents described as an example of one embodiment (for example, components denoted with the same reference signs or names as already described components), the description thereof may be omitted in other embodiments. Contents of operations and processes that are not related to the characteristic features of each embodiment may be omitted. The order of various processes constituting various flows described below is not fixed within a range where no contradiction or the like occurs in the process contents, and the processes may be executed in parallel. In this specification, the terms "first", "second", and "third" are used merely for distinction, and do not indicate the order of processes or the like.

[0014] First Embodiment (Computer Device) As the first embodiment, a computer device that acquires second action set information for an AI agent that performs debugging work is illustrated. FIG. 1 is a diagram illustrating an example of a hardware configuration of a computer device 1 corresponding to at least one embodiment of the present disclosure. The computer device 1 includes, for example, a processor 11, a memory 12, a storage 13, and a communication I / F 14, which are connected to each other via a communication bus.

[0015] The processor 11 performs various calculations and control operations. The processor 11 may be a general-purpose processor such as a CPU (Central Processing Unit), or a dedicated processor such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The memory 12 includes storage devices such as a ROM (Read Only Memory) that stores various programs, and a RAM (Random Access Memory) that provides a work area when various programs are executed.

[0016] Storage 13 includes, for example, storage devices such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives). Communication I / F 14 includes various processing circuits and connection terminals for communicating with external devices via a communication network, and is configured to conform to various communication standards.

[0017] Figure 2 is a diagram showing an example of the functional configuration of a computer device 1 corresponding to at least one embodiment of the present disclosure. As shown in Figure 2, the computer device 1 can communicate with an external device 6 via a communication network 4. The manner of communication connection is not particularly limited. The computer device 1 comprises a storage unit 20, a display device 30, and a control unit 40.

[0018] The memory unit 20 is composed of, for example, a ROM or storage 13. The memory unit 20 stores, for example, a program 21A, a game program 22, a first trained model 23, first image information 24, second image information 25, third image information 26, first action set information 27, and second action set information 28.

[0019] Program 21A is, for example, an information processing program for implementing various functions described later on the computer device 1. Game program 22 is a program for executing a predetermined game. The game is not particularly limited and can be any type of game.

[0020] The first trained model 23 is, for example, a trained model capable of inputting operations to a game related to the game program 22. The first trained model 23 may be, for example, an autonomous AI agent. The first trained model 23 may be capable of, for example, making random operation inputs to the game. Alternatively, the first trained model 23 may be capable of making operation inputs to the game based on predetermined rules (for example, performing a number of predetermined operation inputs in sequence).

[0021] Here, the predetermined rules may be created using, for example, the first pre-trained model 23, the language model 60, or other language models. Specifically, the predetermined rules may be defined by inputting image information or text information representing the state of the game before the change (for example, the first state described later) and the predetermined state after the change (for example, the second state described later) to one of the pre-trained models, and commanding (prompting) it to output the set of actions necessary for this state change, thereby obtaining an action set containing the operations included in the action set. Note that the action set obtained by such a method is not guaranteed to actually cause the above state change, so when such a method is adopted, the trial unit 44 described later plays a role in verifying this action set. That is, multiple candidate action sets that cause the predetermined state change are obtained by such a method, and the trial unit 44 selects the action set that actually causes the predetermined state change from among the multiple candidates. With such a method, for example, an action set that causes the predetermined state change can be obtained more efficiently than having the first pre-trained model 23 input random operations.

[0022] The first image information 24 is, for example, information about the first image that is output when the game state is in the first state. The first image information 24 may be image data of the first image, or it may be text information created based on the first image.

[0023] The second image information 25 is, for example, information about a second image that is output when the game state changes to a second state, which is a state in which a predetermined change has occurred from the first state. The second image information 25 may be image data of the second image, or it may be text information created based on the second image. Various terms such as "first state" and "second state" will be explained together at the end of this embodiment.

[0024] The third image information 26 is, for example, information about a third image that may be output in the second state and is different from the second image. The third image information 26 may be image data of the third image, or it may be text information created based on the third image. The third image information 26 may be stored in the storage unit 20 in advance by the user of the computer device 1. Alternatively, the third image information 26 may be created by converting, for example, test code for game quality control into text or images using a trained model such as the language model 60.

[0025] The text information created based on the first image may, for example, be an extraction of at least one portion of the text contained in the first image, or it may be a textual description of the situation shown in the first image. The method for creating this text information is not particularly limited; for example, known techniques for extracting text from images, such as OCR (Optical Character Reader), may be used, or it may be created using the first trained model 23, the language model 60, or other language models. The same applies to the text information created based on the second image and the text information created based on the third image.

[0026] The first action set information 27 is information indicating the content of at least one operation input when, for example, the game state changes to a second state, which is a state in which a predetermined change has occurred from the first state, due to at least one operation input by the trial unit 44 described later.

[0027] The second action set information 28 is, for example, information indicating the actions that are presumed to transition the game from the first state to the second state. The second action set information 28 is acquired, for example, by the acquisition unit 47 described later.

[0028] Here, using Figure 3, specific examples of the first image information 24, the second image information 25, and the first action set information 27 will be explained. Figure 3 is a diagram showing an example of a predetermined change in a game, corresponding to at least one embodiment of the present disclosure.

[0029] The first image 111 is shown at the top of Figure 3. In the example in Figure 3, the first state is the state in which the menu screen for equipment items is opened, and the equipped and possessed status of each character's equipment items is as shown in the first image 111. In the first image 111, cursor C is at position P1, which is labeled "Equip". For example, if you input "→ (=right button)" from the state shown in the first image 111, cursor C moves to position P2, which is labeled "Exchange". Also, for example, if you input "→,→" from the state shown in the first image 111, cursor C moves to position P3, which is labeled "Discard".

[0030] Furthermore, in the first image 111, "E-" indicates that the equipment item listed after it is equipped. For example, if you input "A (=A button)" from the state shown in the first image 111, cursor C will move to the equipment item slot in the lower row, and you will then be able to equip or unequip the selected equipment item.

[0031] The second image 112 is shown at the bottom of Figure 3. In the second image 112, the "Shakujou" possessed by "Senshi" and the "Mithril Sword" possessed by "Monk" have been swapped from the first state shown in the first image 111. Thus, in the example in Figure 3, the second state is defined as the state in which some change has occurred in the equipment items possessed by any of the characters from the first state shown in the first image 111.

[0032] In the example in Figure 3, the first image information 24 may be, for example, the first image 111, which is image data. Alternatively, the first image information 24 may be text information that includes, for example, character names such as "Akama" and "Monku" contained in the first image 111, and the text of each equipment item name (including "E-"). Similarly, the second image information 25 may be, for example, the second image 112, which is image data. Alternatively, the second image information 25 may be text information that includes, for example, character names such as "Akama" and "Monku" contained in the second image 112, and the text of each equipment item name.

[0033] Furthermore, the first action set information 27 represents the content of the operation input performed by the first trained model 23 during the transition from the first state to the second state shown in Figure 3. In the example in Figure 3, the second image information 25 corresponding to the second image 112 and the first action set information 27 shown in Figure 3 are stored in the storage unit 20 in association. Similarly, the first image information 24 corresponding to the first image 111 may be stored in the storage unit 20 in association with the second image information 25 corresponding to the second image 112 and the first action set information 27 shown in Figure 3.

[0034] Returning to the explanation of Figure 2, the display device 30 is, for example, a device that displays various images output from the computer device 1. The display device 30 may be built into the computer device 1 or it may be an external device of the computer device 1.

[0035] The control unit 40 is composed of, for example, a processor 11 and RAM. When program 21A is executed by the processor 11, the control unit 40 functions as, for example, a communication unit 41, a game execution unit 42, a transition unit 43, a trial unit 44, a recording unit 45, a repeat unit 46, an acquisition unit 47, and a debugging unit 50.

[0036] The communication unit 41 has the function of sending and receiving data to and from other computer devices, such as an external device 6. The game execution unit 42 has the function of running the game by starting the game program 22. Preferably, the game execution unit 42 has the function of saving the game status and loading the saved data. The game execution unit 42 may be configured by an emulator that starts the game program 22 on the computer device 1, for example.

[0037] The transition unit 43 has a function to transition the game state to a first state in which a first image is output. For example, information indicating the operation content for transitioning the game state to the first state may be stored in the storage unit 20 in advance, and the transition unit 43 may be configured to use this information to cause the first trained model 23 to transition to the first state. Alternatively, the transition unit 43 may transition the game state to the first state using, for example, the save function and load function of the game execution unit 42. In this case, the transition unit 43 may be configured to load save data in the first state into the first trained model 23.

[0038] The trial unit 44 has the function of attempting random or predetermined rule-based operation inputs to a game in a first state using, for example, a first trained model 23 capable of inputting operations to the game.

[0039] Furthermore, it is preferable that the trial unit 44 can determine whether or not the system has transitioned to the second state as a result of attempting an operation input. The determination of whether or not the system has transitioned to the second state can be made, for example, by comparing and analyzing the image information or text information of the first image showing the first state with the image information or text information of the image after the operation input by the trial unit 44.

[0040] Furthermore, it is preferable that the trial unit 44 can determine whether or not predetermined transition conditions are met by attempting operation input. Transition conditions can be set appropriately depending on what is being debugged. For example, if the example shown in Figure 3 is adopted as the first and second states, the transition condition can be set as the transition from the menu screen related to equipment items to another screen (for example, the character status screen or the field screen). Alternatively, the transition condition may be set as the case where the second state has not been reached even after the number of operation inputs made by the trial unit 44 exceeds a predetermined number (for example, the number of button inputs is 100 or more). The determination of whether or not the transition conditions are met can be made, for example, by comparing and analyzing the image information or text information of the first image showing the first state with the image information or text information of the image after operation input by the trial unit 44.

[0041] The recording unit 45 has a function to record, for example, when the game state changes to a second state, which is a predetermined change from the first state, due to at least one operation input by the trial unit 44, by associating it with first action set information 27 indicating the content of the at least one operation input and second image information 25 relating to the second image output in the second state. In addition, the recording unit 45 may further record first image information 24 in the storage unit 20, for example, in association with the first action set information 27 and the second image information 25.

[0042] The repeating unit 46 has the function of repeating a series of operations by the transition unit 43, the trial unit 44, and the recording unit 45 until a predetermined termination condition is met. The termination condition is not particularly limited, but examples include the number of first action set information 27 and second image information 25 recorded by the recording unit 45 reaching a predetermined number, or a predetermined time elapsed since the transition to the first state was first made by the transition unit 43.

[0043] The acquisition unit 47 has the function of acquiring at least one second action set of information relating to an operation that is presumed to transition the game from a first state to a second state, based on a plurality of first action set information and a plurality of second image information recorded by the recording unit 45.

[0044] The acquisition unit 47 includes, for example, a first acquisition unit 48. The first acquisition unit 48 inputs, for example, a plurality of first action set information and a plurality of second image information, and third image information relating to a third image which is an image that may be output in the second state and is different from the plurality of second images, into a predetermined language model, and has the function of acquiring second action set information from the predetermined language model that indicates the operation content that is presumed to transition the game from the first state to the second state in which the third image is output. Preferably, the second action set information 28 acquired by the acquisition unit 47 is recorded in the storage unit 20 in association with the corresponding third image information 26.

[0045] Furthermore, the acquisition unit 47 may include, for example, a second acquisition unit 49 in place of or in addition to the first acquisition unit 48. The second acquisition unit 49 has the function of, for example, inputting a plurality of first action set information 27 and a plurality of second image information 25 into a predetermined language model 60, acquiring a function from the predetermined language model 60 that defines an operation content that is presumed to transition the game state from a first state to a second state, and using the function to acquire second action set information 28 that is different from the first action set information 27.

[0046] The debugging unit 50 has a function to perform debugging work on a game program 22, for example. The debugging unit 50 performs debugging work by executing the operations shown in the second action set information 28 acquired by the first acquisition unit 48 and / or the second acquisition unit 49. The debugging work may be performed using, for example, the first trained model 23, or using a general-purpose autonomous AI agent for debugging (a trained model different from the first trained model 23).

[0047] External device 6 is a computer device different from computer device 1. External device 6 may be a device managed by the same administrator as the user of computer device 1, or it may be a device managed by a different administrator than the user of computer device 1.

[0048] The external device 6 is equipped with a predetermined language model 60. The language model 60 may be a so-called large-scale language model (LLM) or a so-called small-scale language model (SLM). As the language model 60, an existing generative AI model such as GPT-4o may be used. In this embodiment, the external device 6 is equipped with the language model 60, but the computer device 1 may be configured to be equipped with the language model 60.

[0049] (Information processing methods) Figure 4 is a flowchart illustrating an example of an information processing method corresponding to at least one embodiment of the present disclosure. As shown in Figure 4, in step S1, the computer device 1 starts the game according to the game program 22 and executes the game. Next, the computer device 1 repeats the processes in steps S2 to S6 until a predetermined termination condition is met.

[0050] In step S2, the computer device 1 transitions the game state to the first state. In step S2, the first trained model 23 executes, for example, operations that are pre-stored as operations to transition to the first state, or operations to load save data that is in the first state.

[0051] In step S3, the computer device 1 uses the first trained model 23 to attempt random or rule-based inputs for the game in the first state. The process in step S3 continues, for example, until the game state becomes the second state or the transition conditions are met by the process in step S3.

[0052] If the game state becomes the second state as a result of the processing in step S3 (YES in step S4), proceed to step S6. If the game state does not become the second state as a result of the processing in step S3 but the transition condition is met (NO in step S4 and YES in step S5), return to step S2.

[0053] In step S6, the computer device 1 stores information indicating the content of the series of operation inputs attempted in step S3 as first action set information 27 in the storage unit 20. In addition, in association with the first action set information 27, it records the image of the game that has entered the second state as a result of the series of operation inputs attempted in step S3 as second image information 25. Also in step S6, the computer device 1 records the first image information 24, which is information about the first image immediately after transitioning to the first state in step S2, in association with the first action set information 27 and the second image information 25. Note that if the first image immediately after transitioning to the first state in step S2 is the same each time, the first image information 24 may be recorded only the first time. If the termination condition is not met, the process returns to step S2 after the processing in step S6.

[0054] In step S7, the computer device 1 obtains at least one second action set information 28 relating to an operation that is presumed to transition the game from a first state to a second state, based on the plurality of first action set information 27 recorded in step S6 and the plurality of second image information 25 associated with each of the plurality of first action set information 27. The processing in step S7 will be described in detail in a later paragraph.

[0055] In step S8, the computer device 1 debugs the game using the second action set information 28 acquired in step S7. Specifically, it instructs the first trained model 23 or another autonomous AI agent to perform the operations indicated in the second action set information 28 for the game. In step S8, it may also verify whether the result of these operations leads to a transition to the second state, or whether the state becomes the same as the state indicated by the third image information 26.

[0056] Here, we will describe step S7 in detail using Figures 5 to 8. In step S7, only the first acquisition process shown in Figures 5 and 6 may be performed, only the second acquisition process shown in Figures 7 and 8 may be performed, or both the first and second acquisition processes may be performed.

[0057] Figure 5 is a flowchart showing an example of a first acquisition process corresponding to at least one embodiment of the present disclosure. As shown in Figure 5, in step S11, the computer device 1 generates a prompt that includes at least a plurality of first action set information 27 and a plurality of second image information 25 recorded in step S6, and third image information 26 relating to a third image.

[0058] The prompt may include, for example, a statement indicating that the user will transition to the second state indicated by the second image information 25 associated with the first action set information 27 by performing the operation input indicated by the first action set information 27, and a request for the output of an action set to transition to the state indicated by the third image information 26. The prompt may also include the first image information 24. In this case, for example, the prompt may include a statement indicating that the user will transition to the second state indicated by the second image information 25 by performing the operation input indicated by the first action set information 27 in the first state indicated by the first image information 24. The prompt may be created, for example, by adding the above information to a predefined phrase, or it may be generated each time using the first trained model 23, etc.

[0059] In step S12, the computer device 1 sends the prompt generated in step S11 to the external device 6 as input information to be input to the language model 60. In step S13, the language model 60 outputs information indicating the operation content corresponding to the third image, that is, the operation content that is presumed to transition the game from the first state to the second state that outputs the third image, based on the received prompt.

[0060] In step S14, the external device 6 transmits the information output in step S13 to the computer device 1. In step S15, the computer device 1 receives the information transmitted from the external device 6 in step S14 and stores it as the second action set information 28. In step S14, the second action set information 28 is recorded in association with the corresponding third image information 26.

[0061] Figure 6 shows an example of data input to and output from the language model in the first acquisition process shown in Figure 5. In the example in Figure 6, the first state is, for example, the state in which the equipment change screen is opened when each character has no equipment items equipped. The second state in the example in Figure 6 is the state in which a change has occurred from the first state, in which one of the characters has an equipment item equipped.

[0062] The "Warrior["","","","]" in the first image information 24 shown in Figure 6 indicates that the "Warrior" has four slots where equipment items can be equipped, and that no equipment items are equipped in any of those four slots. Also, "Shiroma["Staff","","]""]" indicates that the equipment item "Staff" is equipped in one of the slots where "Shiroma" can equip equipment items. Furthermore, the first action set information 27 shown in Figure 6 contains the sequence of button operations during the transition from this first state to the second state (information about which buttons were operated and the order in which those button operations were performed).

[0063] Figure 6 illustrates a group of specific examples containing multiple examples with the first image information 24, the second image information 25, and the first action set information 27, as well as an inference target containing the first image information 24 and the third image information 26, as data to be input to the language model 60. Other specific examples included in the group of specific examples are not particularly limited, as long as they are examples in which any of the characters is equipped with any of the equipment items. Note that the first image information 24, the second image information 25, and the third image information 26 shown in Figure 6 are text information extracted from a portion of the text contained in the first image, the second image, and the third image, respectively.

[0064] Furthermore, Figure 6 illustrates a second action set information 28, which includes a sequence of button operations that are presumed to transition the game state from the first state indicated by the first image information 24 to the second state indicated by the third image information 26, as data output from the language model 60. Alternatively, a group of inference targets containing multiple inference targets may be included as input data to the language model 60, and multiple second action set information 28 corresponding to multiple third image information 26s may be output from the language model 60.

[0065] Figure 7 is a flowchart showing an example of a second acquisition process corresponding to at least one embodiment of the present disclosure. As shown in Figure 7, in step S21, the computer device 1 generates a prompt which includes at least a plurality of first action set information 27 and a plurality of second image information 25 recorded in step S6.

[0066] The prompt may include, for example, a statement indicating that the user will transition to the second state indicated by the second image information 25 associated with the first action set information 27 by performing the operation input indicated by the first action set information 27, and a request for output of a program function that defines the operation to transition from the first state to the second state. The prompt may also include the first image information 24. In this case, for example, the prompt may include a statement indicating that the user will transition to the second state indicated by the second image information 25 by performing the operation input indicated by the first action set information 27 in the first state indicated by the first image information 24. The prompt may also include text information defining the second state, such as, "The second state is a state in which any character is equipped with any equipment item." The prompt may be created, for example, by adding the above information to a predefined phrase, or it may be generated each time using the first trained model 23, etc.

[0067] In step S22, the computer device 1 sends the prompt generated in step S21 to the external device 6 as input information to be input to the language model 60. In step S23, the language model 60 outputs a program function to transition the game from the first state to the second state based on the received prompt.

[0068] In step S24, the external device 6 transmits information about the function output in step S23 to the computer device 1. In step S25, the computer device 1 receives the information about the function transmitted from the external device 6 in step S24 and stores the information in the storage unit 20.

[0069] In step S26, the computer device 1 obtains the second action set information 28 using the function obtained in step S25. In step S25, for example, the information obtained by inputting appropriate variables into the obtained function is acquired as the second action set information 28. In step S26, multiple sets of second action set information 28 can be obtained by changing the variables substituted into the obtained function.

[0070] Figure 8 shows an example of data input to the language model and data output from the language model in the second acquisition process shown in Figure 7. The first and second states in the example in Figure 8 are the same as in the example in Figure 6.

[0071] Figure 8 illustrates a set of concrete examples containing multiple concrete examples, each including the first image information 24, the second image information 25, and the first action set information 27, as data input to the language model 60. Figure 8 also illustrates a program function for transitioning from the first state to the second state as data output from the language model 60. Figure 8 shows an example where the language model 60 outputs a generalized function for equipping "X" with "Y," derived from examples such as equipping "Shiroma" with a "staff" and other examples (for example, equipping "Senshi" with a "longsword").

[0072] One aspect of the first embodiment is that the man-hours required for debugging can be reduced. More specifically, one aspect of the first embodiment is that, for example, while reducing the man-hours required for debugging, the AI ​​can be made to perform meaningful and complex actions during the debugging process, thereby improving the accuracy of the debugging process.

[0073] Furthermore, as one aspect of the first embodiment, by converting the second and third image information into text information, it is possible to eliminate information that could be considered noise in the image (for example, the position of cursor C shown in Figure 3), thereby improving the accuracy of determining whether or not a transition to the second state has occurred, and improving the accuracy of the second action set information output from the language model.

[0074] Furthermore, as one aspect of the first embodiment, by obtaining a function that defines the operations that are presumed to transition the game state from the first state to the second state, a large amount of second action set information can be obtained without having to prepare third image information in advance.

[0075] In the first embodiment, the "first state" can vary depending on the target of the debugging work. If the target of the debugging work is the behavior when equipping equipment items, the "first state" may be, for example, the state in which the menu screen for equipping equipment items is opened, or the state in which the menu screen is open and the equipment status of each character's equipment items is in a specific state. If the target of the debugging work is the behavior when purchasing items in a shop, the "first state" may be, for example, the state in which the screen for purchasing items in that shop is opened. If the target of the debugging work is the behavior in battle, the "first state" may be, for example, the state in which a battle with a specific enemy has started.

[0076] The "second state" is, for example, a state in which a predetermined change has occurred from the "first state". The "second state" can change depending on the target of the debugging work. If the target of the debugging work is the behavior when equipping an equipment item, the "second state" may be, for example, when the equipment status of any character changes. If the target of the debugging work is the behavior when purchasing an item at a shop, the "second state" may be, for example, the state in which an item has been purchased at that shop. The "predetermined change" can change depending on the target of the debugging work. The changes described above may be considered the "predetermined change," or other changes may be considered the "predetermined change."

[0077] [Second Embodiment] (Computer device) The computer device according to the second embodiment is capable of generating a second trained model trained using training data in which a plurality of first action set information is used as the target variable and a plurality of second image information is used as the explanatory variable. The hardware configuration of the computer device according to the second embodiment can be, for example, the same as that in Figure 1. The computer device according to the second embodiment will be described in detail below with reference to Figures 9 and 10.

[0078] Figure 9 shows an example of the functional configuration of a computer device 101 corresponding to at least one embodiment of the present disclosure. Elements in Figure 9 that are denoted by the same reference numerals as those in Figure 2 are described in the first embodiment, and their descriptions are omitted as appropriate.

[0079] Program 21B is, for example, an information processing program for implementing various functions described later on the computer device 101. The second trained model 29 is a trained model generated by the generation unit 51. When the second trained model 29 receives, for example, third image information 26 (not shown in Figure 9) as described in the first embodiment, it outputs second action set information 28 (not shown in Figure 9) which is presumed to transition from the first state to the second state indicated by the third image information 26. The second trained model 29 may be generated, for example, based on a known large-scale language model or small-scale language model.

[0080] The generation unit 51 has the function of generating a second trained model 29 by training using training data in which a plurality of first action set information 27 recorded by the recording unit 45 is used as the target variable and a plurality of second image information 25 is used as the explanatory variable.

[0081] (Information processing methods) Figure 10 is a flowchart showing an example of an information processing method corresponding to at least one embodiment of the present disclosure. Although the flowchart in Figure 10 includes the processing of steps S1 to S6 and S31, the explanation of steps S1 to S6 will be omitted as it will refer to the content explained in the first embodiment.

[0082] In step S31, the computer device 1 generates a second trained model 29 using the multiple first action set information 27 recorded in step S6 and the multiple second image information 25 associated with each of the multiple first action set information 27 as training data. In step S31, training is performed with the multiple first action set information 27 as the target variable and the multiple second image information 25 as the explanatory variables.

[0083] One aspect of the second embodiment is that the man-hours required for debugging can be reduced. More specifically, one aspect of the second embodiment is that, for example, while reducing the man-hours required for debugging, it becomes possible to have the AI ​​perform meaningful and complex actions during the debugging process, thereby improving the accuracy of the debugging process.

[0084] In the second embodiment, the "first state," "second state," and "predetermined change" can be the same as those described in the first embodiment, to the extent necessary. In the second embodiment, "learning" refers to, for example, machine learning. Examples of machine learning include supervised learning using deep learning, but unsupervised learning or other learning methods such as reinforcement learning may also be used. Furthermore, "learning" may be pre-training using a large amount of data, or it may be fine-tuning or transfer learning tailored to the purpose or field.

[0085] [Third Embodiment] (Computer device) The computer device according to the third embodiment acquires the second action set information 28 using, for example, the second trained model 29 described in the second embodiment. The hardware configuration of the computer device according to the third embodiment can be, for example, the same as that in Figure 1. The computer device according to the third embodiment will be described in detail below with reference to Figures 11 to 13.

[0086] Figure 11 is a diagram showing an example of the functional configuration of a computer device 201 corresponding to at least one embodiment of the present disclosure. Elements in Figure 11 that are denoted by the same reference numerals as those in Figure 2 or Figure 9 are described in the first and second embodiments, and their descriptions are omitted as appropriate.

[0087] Program 21C is, for example, an information processing program for implementing various functions described later on the computer device 201. The third acquisition unit 52 has the function of inputting the third image information 26 into the second trained model 29 and causing the second trained model 29 to output second action set information 28 indicating the operations that are presumed to transition the game from the first state to the second state in which the third image shown in the third image information 26 is displayed, thereby acquiring the second action set information 28.

[0088] (Information processing methods) Figure 12 is a flowchart showing an example of an information processing method (third acquisition process) corresponding to at least one embodiment of the present disclosure. As shown in Figure 12, in step S41, the computer device 201 inputs the third image information 26 to the second trained model 29. In step S42, the computer device 201 acquires the information output from the second trained model 29 and stores it as the second action set information 28.

[0089] Figure 13 shows an example of the data input to the second trained model 29 and the data output from the second trained model 29 in the third acquisition process shown in Figure 12. The first and second states in the example in Figure 13 are the same as in the example in Figure 6.

[0090] Figure 13 illustrates an inference target including the first image information 24 and the third image information 26 as data input to the second trained model 29. Also in Figure 13, a second action set information 28, which includes a sequence of button operations that are presumed to transition the game state from the first state indicated by the first image information 24 to the second state indicated by the third image information 26, is illustrated as data output from the second trained model 29. Alternatively, a group of inference targets containing multiple inference targets may be included as input data to the second trained model 29, and multiple second action set information 28 corresponding to multiple third image information 26s may be output from the second trained model 29.

[0091] One aspect of the third embodiment is that the man-hours required for debugging can be reduced. More specifically, one aspect of the third embodiment is that, for example, while reducing the man-hours required for debugging, it becomes possible to have the AI ​​perform meaningful and complex actions during the debugging process, thereby improving the accuracy of the debugging process.

[0092] In the third embodiment, the "first state," "second state," and "predetermined change" can each be the same as those described in the first embodiment, to the extent necessary. In the third embodiment, the "learning" can be the same as those described in the second embodiment, to the extent necessary.

[0093] Although the embodiments described herein have been described in detail above, the present invention is not limited to the embodiments described above, and can be modified, improved, etc., as appropriate. The present invention includes all modifications within the meaning and scope of the claims and equivalents thereof.

[0094] [Note] The above description of the embodiments is written in such a way that a person with ordinary skill in the art to which the invention pertains can implement the invention described below.

[0095] (Item 1) In a computer device, A transition function that moves the game state to the first state where the first image is output. A trial function that attempts to input random or predetermined rules of operation to the game in the first state using a first trained model capable of inputting operations to the game, When the state of the game changes to a second state, which is a predetermined change from the first state, as a result of at least one operation input by the trial function, a recording function records a first action set information indicating the content of the at least one operation input and a second image information relating to the second image output in the second state. A repeat function that repeats the series of operations performed by the transition function, the trial function, and the recording function until a predetermined termination condition is met. Based on the plurality of first action set information and plurality of second image information recorded by the recording function, an acquisition function is realized to acquire at least one second action set information relating to an operation that is presumed to transition the game from a first state to a second state. The acquisition function includes a first acquisition function that inputs a plurality of first action set information and a plurality of second image information and a third image information relating to a third image which is an image that can be output in the second state and is different from the plurality of second images into a predetermined language model, and acquires from the predetermined language model second action set information indicating an operation that is presumed to transition the game from the first state to the second state in which the third image is output. program.

[0096] (Item 2) The acquisition function described above may be used in place of or in addition to the first acquisition function described above. The system includes a second acquisition function which inputs a plurality of first action set information and a plurality of second image information into a predetermined language model, obtains a function from the predetermined language model that defines an operation that is presumed to transition the game state from a first state to a second state, and uses the function to obtain second action set information that is different from the first action set information. The program described in item 1.

[0097] (Item 3) The second image information is text information obtained by extracting at least one portion of the text contained in the second image. When inputting the third image information into the predetermined language model, the third image information is text information obtained by extracting at least one portion of the text contained in the third image. The program described in item 1 or item 2.

[0098] (Item 4) Computer equipment, A transition step that moves the game state to the first state where the first image is output, A trial step in which a first trained model capable of inputting operations to the game is used to attempt random or predetermined rule-based operation inputs to the game in the first state, If, as a result of at least one operation input in the trial step, the state of the game changes to a second state which is a predetermined change from the first state, a recording step is performed to record, in association with, first action set information indicating the content of the operation input and second image information relating to the second image output in the second state. A repeat step which repeats the series of operations in the transition step, the trial step, and the recording step until a predetermined termination condition is met, The process includes: an acquisition step of acquiring at least one second action set of information relating to an operation that is presumed to transition the game from a first state to a second state, based on a plurality of first action set pieces of information and a plurality of second image pieces of information recorded in the recording step; The acquisition step includes inputting a plurality of first action set information and a plurality of second image information and a third image information relating to a third image which is an image that may be output in the second state and is different from the plurality of second images into a predetermined language model, and acquiring second action set information from the predetermined language model that indicates an operation that is presumed to transition the game from the first state to the second state which outputs the third image. Information processing methods.

[0099] (Item 5) The acquisition step may be performed in place of or in addition to the first acquisition step. The process includes a second acquisition step of inputting a plurality of first action set information and a plurality of second image information into a predetermined language model, obtaining a function from the predetermined language model that defines an operation that is presumed to transition the game state from a first state to a second state, and using the function to obtain second action set information that is different from the first action set information. The information processing method described in item 4.

[0100] (Item 6) The second image information is text information created based on the second image, When inputting the third image information into the predetermined language model, the third image information is text information created based on the third image. The information processing method described in item 4 or item 5.

[0101] (Item 7) Computer equipment, A transition step that moves the game state to the first state where the first image is output, A trial step in which a first trained model capable of inputting operations to the game is used to attempt random or predetermined rule-based operation inputs to the game in the first state, If, as a result of at least one operation input in the trial step, the state of the game changes to a second state which is a predetermined change from the first state, a recording step is performed to record, in association with, first action set information indicating the content of the operation input and second image information relating to the second image output in the second state. A repeat step which repeats the series of operations in the transition step, the trial step, and the recording step until a predetermined termination condition is met, The process includes: a generation step of generating a second trained model trained using training data in which a plurality of first action set information recorded in the recording step is used as the target variable and a plurality of second image information are used as the explanatory variables; and a generation step of generating a second trained model trained using this training data. Method for generating pre-trained models.

[0102] (Item 8) The second image information is text information created based on the second image. The method for generating a pre-trained model as described in item 7.

[0103] (Item 9) The second trained model generated by the trained model generation method described in Item 7 or Item 8 is made available on a computer device. A third acquisition function is implemented that inputs third image information relating to a third image, which is an image that may be output in the second state, into the second trained model, and acquires second action set information from the second trained model that indicates the operations that are presumed to transition the game from the first state to the second state where the third image is output. program.

[0104] (Item 10) The second trained model generated by the trained model generation method described in Item 7 or Item 8 is placed on an available computer device. A third acquisition function is implemented that inputs third image information relating to a third image, which is an image that may be output in the second state, into the second trained model, and acquires second action set information from the second trained model that indicates the operations that are presumed to transition the game from the first state to the second state where the third image is output. program.

[0105] (Item 11) The third image information is text information created based on the third image. The program described in item 10. [Explanation of symbols]

[0106] 1,101,201: Computer device 6: External device

Claims

1. In a computer device, A transition function that moves the game state to the first state where the first image is output. A trial function that attempts to input random or predetermined rules of operation to the game in the first state using a first trained model capable of inputting operations to the game, When the state of the game changes to a second state, which is a state in which a predetermined change has occurred in the first state, due to at least one operation input by the trial function, a recording function records a first action set information indicating the content of the operation input and a second image information relating to the second image output in the second state, in association with each other. A repeat function that repeats the series of operations performed by the transition function, the trial function, and the recording function until a predetermined termination condition is met. Based on the plurality of first action set information and the plurality of second image information recorded by the recording function, an acquisition function is realized to acquire at least one second action set information relating to an operation that is presumed to transition the game from a first state to a second state. The acquisition function includes a first acquisition function that inputs a plurality of first action set information and a plurality of second image information and a third image information relating to a third image which is an image that can be output in the second state and is different from the plurality of second images into a predetermined language model, and acquires from the predetermined language model the second action set information which indicates an operation that is presumed to transition the game from the first state to the second state in which the third image is output. program.

2. The acquisition function may be provided in place of or in addition to the first acquisition function. The system includes a second acquisition function which inputs a plurality of first action set information and a plurality of second image information into a predetermined language model, obtains a function from the predetermined language model that defines an operation that is presumed to transition the game state from a first state to a second state, and uses the function to obtain second action set information that is different from the first action set information. The program according to claim 1.

3. The second image information is text information created based on the second image. When inputting the third image information into the predetermined language model, the third image information is text information created based on the third image. The program according to claim 1 or 2.

4. In a computer device, A transition step that moves the game state to the first state where the first image is output, A trial step in which a first trained model capable of inputting operations to the game is used to attempt random or predetermined rule-based operation inputs to the game in the first state, If, as a result of at least one operation input in the trial step, the state of the game changes to a second state in which a predetermined change has occurred from the first state, a recording step is performed to record, in association with, first action set information indicating the content of the operation input at least one operation input and second image information relating to the second image output in the second state. A repeat step which repeats the series of operations in the transition step, the trial step, and the recording step until a predetermined termination condition is met, The recording step includes performing an acquisition step to acquire at least one second action set of information relating to an operation that is presumed to transition the game from a first state to a second state, based on a plurality of first action set information and a plurality of second image information recorded in the recording step, The acquisition step includes inputting a plurality of first action set information and a plurality of second image information and a third image information relating to a third image which is an image that can be output in the second state and is different from the plurality of second images into a predetermined language model, and acquiring the second action set information from the predetermined language model which indicates an operation that is presumed to transition the game from the first state to the second state in which the third image is output. Information processing methods.

5. In a computer device, A transition step that moves the game state to the first state where the first image is output, A trial step in which a first trained model capable of inputting operations to the game is used to attempt random or predetermined rule-based operation inputs to the game in the first state, If, as a result of at least one operation input in the trial step, the state of the game changes to a second state in which a predetermined change has occurred from the first state, a recording step is performed to record, in association with, first action set information indicating the content of the operation input at least one operation input and second image information relating to the second image output in the second state. A repeat step which repeats the series of operations in the transition step, the trial step, and the recording step until a predetermined termination condition is met, The process includes: a generation step of generating a second trained model trained using training data in which a plurality of first action set information recorded in the recording step is used as the target variable and a plurality of second image information are used as the explanatory variables; and a generation step of generating a second trained model trained using this training data. Method for generating pre-trained models.

6. The second trained model generated by the trained model generation method described in claim 5 is made available to a computer device. A third acquisition function is implemented which inputs third image information relating to a third image that may be output in the second state into the second trained model, and acquires second action set information from the second trained model that indicates the operations that are presumed to transition the game from the first state to the second state in which the third image is output. program.

Images