Information processing method, program, and information processing system

Abstract

To provide an information processing method, an information processing system, and a program for efficiently obtaining a desired processing result.SOLUTION: A method includes: an input information acquisition step S100 of acquiring input information to be input to a trained model; a version identifier generation step S104 of generating a version identifier for identifying a version of the input information; a storage step S106 of storing the input information and the version identifier corresponding to the input information in a repository; and update steps S108-S114 of generating a new version identifier in response to update of the input information in the repository and storing the updated input information and the version identifier corresponding thereto in the repository.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing method, a program, and an information processing system. [Background technology]

[0002] Patent Document 1 discloses a device that uses a trained model. In this device, a secure component manages parameters and version information of the trained model, and the device deploys the trained model based on input information input to the trained model and executes processing using the trained model. This allows processing using the trained model to be performed while being protected from external attacks. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-101807 Summary of the Invention [Problem to be solved by the invention]

[0004] When performing information processing using a trained model, input information may be adjusted to obtain the desired processing results from the trained model. However, in order to adjust the input information efficiently, it is preferable to manage the change history.

[0005] In consideration of the above, an object of the present invention is to efficiently obtain a desired processing result. [Means for solving the problem]

[0006] According to one embodiment, an information processing method is executed by an information processing device, and includes an input information acquisition step of acquiring input information to be input to a trained model, a version identifier generation step of generating a version identifier for identifying the version of the input information, a storage step of storing the input information and the version identifier corresponding to the input information in a repository, and an update step of, when the input information in the repository is updated, generating a new version identifier in response to the update and storing the updated input information and the version identifier corresponding to the updated input information in the repository.

[0007] According to one embodiment of an information processing method, the information processing method is executed by an information processing device and includes an input information acquisition step of acquiring input information to be input to a trained model and tag information in the input information, and a storage step of storing the input information and the tag information in a repository.

[0008] According to one embodiment of the program, an information processing device is caused to execute an information processing method including an input information acquisition step of acquiring input information to be input to a trained model, a version identifier generation step of generating a version identifier for identifying the version of the input information, a storage step of storing the input information and the version identifier corresponding to the input information in a repository, and an update step of, when the input information in the repository is updated, generating a new version identifier in response to the update and storing the updated input information and the version identifier corresponding to the updated input information in the repository.

[0009] According to one embodiment of the program, an information processing device is caused to execute an information processing method including an input information acquisition step of acquiring input information to be input to a trained model, an input information acquisition step of acquiring input information to be input to the trained model and tag information in the input information, and a storage step of storing the input information and the tag information in a repository.

[0010] According to one embodiment, the information processing system is executed by an information processing device and includes an input information acquisition unit that acquires input information to be input to a trained model, a version identifier generation unit that generates a version identifier for identifying the version of the input information, a storage unit that stores the input information and the version identifier corresponding to the input information in a repository, and an update unit that, when the input information in the repository is updated, generates a new version identifier in response to the update and stores the updated input information and the version identifier corresponding to the updated input information in the repository.

[0011] According to one embodiment, the information processing system is executed by an information processing device and includes an input information acquisition unit that acquires input information to be input to a trained model, an input information acquisition unit that acquires input information to be input to the trained model and tag information in the input information, and a storage unit that stores the input information and the tag information in a repository. [Effects of the Invention]

[0012] According to one embodiment, desired processing results can be obtained efficiently. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to the first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a server according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a processing flow of the information processing system according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a functional configuration of a server according to a second embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a processing flow of an information processing system according to a second embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of a functional configuration of a server according to the third embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of a processing flow of an information processing system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] (First embodiment) A first embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 4. In each drawing, the same or equivalent components and parts are denoted by the same reference numerals. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0015] (System Overview) First, an overview of an information processing system 10 according to this embodiment will be described. The information processing system 10 according to this embodiment is a system for managing input information to be input to a trained model. In this embodiment, as an example, this input information is a prompt to be input to a large-scale language model as a trained model. This prompt is based on data such as natural language, video, and audio. The prompt also includes so-called templates, at least a portion of which can be changed or added.

[0016] (System Configuration) Fig. 1 is a diagram showing an example of the configuration of an information processing system 10 according to this embodiment. As shown in Fig. 1, the information processing system 10 according to this embodiment includes a server 12 as an information processing device and a user terminal 14 as an information processing device, which are communicably connected to each other via a network N. The network N is, for example, a wired local area network (LAN), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination thereof.

[0017] The user terminal 14 is an example of an information processing device that is operated by a user U to input and display various information. The user terminal 14 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination of these.

[0018] The server 12 is an example of an information processing device that acquires information input from the user terminal 14, processes the information, and outputs the results. The server 12 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the server 12 will be described later.

[0019] (Hardware configuration) 2 is a block diagram showing the hardware configuration of the server 12. The server 12 includes a processor 120, a memory 122, a storage 124, a communication I / F 126, an input / output I / F 128, and a drive device 134, which are communicatively connected to each other via a bus B.

[0020] The processor 120 controls each component of the server 12 and realizes the functions of the server 12 by loading various programs stored in the storage 124 into the memory 122 and executing them. The programs executed by the processor 120 include, but are not limited to, an operating system (OS) and a program 220 described below. Execution of these programs by the processor 120 realizes part of the state visualization method according to this embodiment. The processor 120 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), or a combination thereof.

[0021] The memory 122 is, for example, a read-only memory (ROM), a random access memory (RAM), or a combination thereof. The ROM is, for example, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a combination thereof. The RAM is, for example, a dynamic random access memory (DRAM), a static random access memory (SRAM), a magnetoresistive random access memory (MRAM), or a combination thereof.

[0022] The storage 124 stores the OS, various programs described below, and various data. The storage 124 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

[0023] The communication I / F 126 is an interface for connecting the server 12 to external devices including the user terminal 14 via the network N and controlling communication. The communication I / F 126 is, for example, an adapter compliant with Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (e.g., Fibre Channel), but is not limited to these.

[0024] The input / output I / F 128 is an interface for connecting an input device 132 and an output device 130 to the server 12. The input device 132 is, for example, a mouse, a keyboard, a touch panel, a microphone, a scanner, a camera, various sensors, an operation button, or a combination thereof. The output device 130 is, for example, a display, a projector, a printer, a speaker, a vibrator, or a combination thereof.

[0025] The drive device 134 reads and writes data from and to the disk media 136. The drive device 134 is, for example, a magnetic disk drive, an optical disk drive, a magneto-optical disk drive, or a combination thereof. The disk media 136 is, for example, a compact disc (CD), a digital versatile disc (DVD), a floppy disk (FD), a magneto-optical disk (MO), a Blu-ray (registered trademark) disc (BD), or a combination thereof.

[0026] In this embodiment, the program may be written into memory 122 or storage 124 during the manufacturing stage of server 12, may be provided to server 12 via network N, or may be provided to server 12 via a non-transitory computer-readable recording medium such as disk media 136.

[0027] Furthermore, the hardware configuration of the user terminal 14 is substantially the same as the hardware configuration of the server 12 described above, and therefore a detailed description thereof will be omitted.

[0028] (Functional configuration) Next, the functional configuration of the server 12 will be described. Fig. 3 is a diagram showing an example of the functional configuration of the server 12. When executing various programs, the server 12 uses the above-mentioned hardware resources to realize various functions. The server 12 has a communication unit 20, a storage unit 22, and a control unit 24 as the functional configuration realized by the server 12. Each functional configuration is realized by the processor 120 reading and executing a program 220 stored in the memory 122 or the storage 124.

[0029] The communication unit 20 is realized by the communication I / F 126. The communication unit 20 transmits and receives information to and from the user terminal 14 via the network N. The communication unit 20 receives information input from the user terminal 14. The communication unit 20 also transmits information to the user terminal 14 and receives requests from the user U from the user terminal 14. Furthermore, the communication unit 20 transmits and receives information to and from external systems using APIs, etc.

[0030] The storage unit 22 is realized by the memory 122 and the storage 124. The storage unit 22 stores a program 220, a trained model 222, a prompt DB 224, a user information DB 226, and a processing result DB 228.

[0031] The trained model 222 is composed of at least one trained machine learning model. As described above, the trained model 222 is, for example, a large-scale language model trained on a large amount of text data from internet articles, books, websites, etc. When text data called a prompt is input as input information, the trained model 222 executes information processing in accordance with the prompt, generates various data including the text data as a result, and outputs the data as output information. Note that the trained models 222 may be stored not in the storage unit 22 but on an external server or the like that can communicate with the server.

[0032] The trained model 222 also generates additional information as needed for prompt templates, which will be described later. That is, when the content of a prompt template is updated and the prompt template itself is evaluated by comparing and evaluating the output information of the pre-update and post-update prompt templates, additional information is generated according to the template. For example, if a prompt template states, "Please translate as follows: Source language: , Translated language: , Tone of the translated text (business, casual, easy to understand for general users, etc.): , Text to be translated: ," placeholders are set for the "Source language," "Translated language," "Tone of the translated text (business, casual, easy to understand for general users, etc.)," ​​and "Tenth text to be translated" in this template. These placeholders are temporary spaces or markers provided for the user to enter specific information later. The trained model 222 generates information corresponding to these placeholders as additional information. In the above example, additional information such as "English" for the "source language," "Japanese" for the "translated language," "business-oriented" for the "tone of the translated text (business, casual, easy to understand for general users, etc.)," ​​and "A picture is worth a thousand words" for the "text to be translated" is generated. By adding this additional information to a template and inputting it as a prompt to the trained model 222, output information can be obtained.

[0033] Furthermore, the trained model 222 compares how the output information changes between the prompt (based on a template) before the content is updated and the prompt (based on a template) after the content is updated, and evaluates whether the output information is consistent with the intent of the prompt. In other words, it can determine whether updating the prompt template will result in the desired output information. This comparative evaluation is performed by inputting the output information to the trained model 222 along with a prompt instructing the trained model to perform the evaluation. However, this is not limited to this, and the evaluation may also be performed by an external trained model connected for communication. This allows evaluation from different perspectives and objective evaluation.

[0034] As described above, the input information is a prompt that instructs processing to the trained model 222, and is text data in natural language. This input information is input by the user U, but is not limited to this, and may be automatically generated or modified by the system.

[0035] The prompt DB 224 is a database that stores information indicating prompts input by the user U. Prompts are written in natural language, and the prompt DB 224 stores, as an example, prompts associated with account information (described later) at the time the prompt was input. The prompt DB 224 also stores not only all prompts but also at least some prompts that leave room for input of additional information. At least some of these prompts correspond to prompt templates. A template with additional input information input corresponds to a prompt.

[0036] Version identifiers corresponding to prompts (including templates, the same applies below) are also stored in the prompt DB 224. The version identifier is arbitrary text data for uniquely identifying the version of the prompt, and when the contents of the prompt are updated, a new version identifier (described in detail later) is generated accordingly and stored in the prompt DB 224.

[0037] The user information DB 226 is a database that stores various information such as user U and manager M who have registered to use the information processing system 10. Examples of the various information stored include user identification information for unique identification, user preference information such as interest information, user history information including past usage status, and user attribute information such as occupation, position, and authority.

[0038] The processing result DB228 is a database that stores information indicating the results of executing a process based on a prompt (also referred to as the processing execution result or the processing result). The processing result is various information including text information and image information, and the processing result DB228 stores the processing result, the prompt, and user information in a linked manner. The processing result DB228 also stores at least one of evaluation and comparison of output information obtained by inputting prompts corresponding to multiple different version identifiers into the trained model 222.

[0039] The control unit 24 is realized by the processor 120 reading and executing the program 220 from the memory 122 (see FIG. 2) and working in cooperation with other hardware components. The control unit 24 includes an information acquisition unit 242, an information processing unit 244, and an output unit 246.

[0040] The information acquisition unit 242 acquires various information required for processing by the information processing unit 244. Specifically, it acquires prompts stored in a storage unit including the memory unit 22 or an external database connected for communication, or prompts input by the user U. Note that the various prompts acquired are not limited to text data, and may utilize information from various sensors such as voice input, gestures, hardware operations, eye tracking, and biosensors.

[0041] The information acquisition unit 242 also acquires various information for executing a search for prompts stored in the storage unit 22 or an external, communicatively connected database, etc. That is, the information acquisition unit 242 acquires search execution instruction information and search condition information as various information from the user terminal 14 or an external, communicatively connected system, etc., using an API, etc. This search condition information is based on metadata such as natural language keywords and tags containing the content of the prompt, version identifiers, branch identifiers, branch names (details of which will be described later), creators, and creation dates and times. That is, the information acquisition unit 242 corresponds to the "input information acquisition unit" recited in claims 15 and 16. The search condition information can also be referred to as search keywords, search terms, search phrases, queries, or search input information.

[0042] Based on the prompt, the information processing unit 244 performs information processing using the trained model 222. The trained model 222 outputs an answer or a generated result corresponding to the prompt based on knowledge accumulated therein.

[0043] The information processing unit 244 also constantly monitors the acquired prompt and determines whether the content of the prompt matches an existing prompt stored in the storage unit. For example, this determination is performed by generating a hash value or checksum of the prompt and comparing it with existing prompt data. If a matching prompt does not exist, the prompt is treated as a new prompt. Alternatively, the matching determination may be performed using keyword matching, natural language processing using the trained model 222, feedback information from the user or the system, or the like. If an identical prompt does not exist, the information processing unit 244 generates a unique version identifier for the new prompt. This version identifier is used to identify the prompt. For example, the information processing unit 244 generates an identifier that ensures uniqueness using the current date and time or a random value. The information processing unit 244 then stores the new prompt together with the generated version identifier in the storage unit. The information processing unit 244 is also capable of generating a version identifier for output information from the trained model 222.

[0044] Furthermore, the information processing unit 244 generates additional information corresponding to a placeholder in a prompt template as needed. Specifically, the information processing unit 244 extracts a placeholder in the template and generates additional information to be inserted into the placeholder. The additional information includes dynamically generated data (e.g., the latest information obtained from an external source) and static data (fixed information), and is generated according to a data format specified for the placeholder (e.g., "NAME," "DATE," "TIME," etc.). Note that the template and an instruction to generate additional information corresponding to the placeholder may be input as a prompt to the trained model 222, and the output information may be used to obtain the additional information. The information processing unit 244 may also request the user U to input additional information as needed. This is particularly effective in an interactive system in which the system and the user U converse in a chat format. Then, by inserting the generated additional information into the placeholder, a more specific and customized prompt is formed.

[0045] Furthermore, the information processing unit 244 compares and evaluates output results based on prompts of different versions. This comparative evaluation makes it possible to clarify the differences in performance and usefulness between versions.

[0046] Furthermore, the information processing unit 244 generates and executes a query for searching for prompts in the storage unit based on execution instruction information and search condition information acquired from the user U or the system. The search results are provided to the user U via the user terminal 14 as a list of corresponding prompts and metadata. The information processing unit 244 is also capable of creating a branch in response to the user U's operation for a prompt corresponding to a specific version identifier, and storing the branch information in the prompt DB 224. That is, new input information based on the user U's branch creation operation is created by referencing the specific (original) version identifier, and a new version identifier is generated for that new input information. The generated version identifier is associated with the original version identifier, and at least one of a branch identifier and a branch name is generated as branch information. This information is stored in the prompt DB 224. That is, the information processing unit 244 corresponds to the "version identifier generation unit," "storage execution unit," and "update unit" recited in claims 15 and 16.

[0047] As described above, the information processing unit 244 manages input information, output information, and version identifiers such as prompts and templates, and stores the input information and version identifiers in the prompt DB 224 and the output information in the processing result DB 228. Therefore, the prompt DB 224, processing result DB 228, and information processing unit 244 work together to function as a repository.

[0048] The output unit 246 controls the information processing unit 244 so that the processing result is output to the user terminal 14 .

[0049] (Processing performed by information processing system 10) Next, the operation of the information processing system 10 will be described. Fig. 4 is a flowchart showing an example of the flow of processing by the information processing system 10. The processor 120 reads out the program 220 stored in the storage 124, expands it in the memory 122, and executes it, thereby performing processing. Although not shown, when the processor 120 receives operation information to terminate the operation of the information processing system 10, or operation termination information from the user terminal 14 during the determination processing being executed (these will be simply referred to as "termination operations"), the processor 120 terminates the processing based on the program 220 being processed.

[0050] The processor 120 determines whether or not input information has been acquired (step S100). If input information has not been acquired (step S100: NO), the processor 120 proceeds to step S130, which will be described later. On the other hand, if input information has been acquired (step S100: YES), the processor 120 determines whether the content of the input information matches existing input information stored in the storage unit (step S102). If they match, that is, if the input information is not new (step S102: YES), the processor 120 proceeds to step S108, which will be described later. On the other hand, if they do not match, that is, if the input information is new (step S102: NO), the processor 120 generates a version identifier for the input information (step S104) and stores the input information and the version identifier in the storage unit (step S106). The processing of step S100 described above corresponds to the "input information acquisition step" recited in claim 1. Furthermore, the processing of step S104 corresponds to the "version identifier generation step" recited in claim 1. Furthermore, the process of step S106 corresponds to the "storing step" recited in claim 1.

[0051] The processor 120 monitors whether the input information has been updated (step S108) and determines whether the input information has been updated (step S110). If there has been no update (step S110: NO), the processor 120 proceeds to step S130, which will be described later. On the other hand, if there has been an update (step S110: YES), the processor 120 generates a version identifier for the updated input information (step S112) and stores the updated input information and the version identifier in the storage unit (step S114). The processes from step S108 to step S114 correspond to the "update step" recited in claim 1.

[0052] Processor 120 determines whether additional information is required for the input information (i.e., whether the input information is a template with a placeholder) (step S116). If additional information is not required (step S116: NO), processor 120 proceeds to step S120, which will be described later. On the other hand, if additional information is required (step S116: YES), processor 120 generates additional information (step S118).

[0053] The processor 120 inputs input information to the trained model 222 (step S120) and acquires output information from the trained model 222 (step S122). Then, the processor 120 generates a version identifier for the acquired output information (step S124).

[0054] Processor 120 compares output information corresponding to different versions of input information (step S126). For different versions, the latest version may be compared with the immediately previous version, or any different versions may be compared. Processor 120 evaluates the output information and thus the input information based on the comparison results (step S128), and stores the evaluation results in a storage unit in association with the input information (step S130).

[0055] The processor 120 determines whether or not search execution instruction information and search condition information have been acquired (step S132). If not (step S132: NO), the processor 120 terminates the processing based on the program 220. On the other hand, if acquired (step S132: YES), the processor 120 executes the search (step S134), outputs the search results to the user terminal 14 for display (step S136), and terminates the processing based on the program 220. The processing from step S132 to step S136 corresponds to the "acquisition step" recited in claim 2. The processing from step S132 to step S134 corresponds to the "search step" recited in claim 11. Although not shown, if the user U performs a branch creation operation based on the search results, a branch identifier and a branch name are generated in accordance with the operation and stored in the prompt DB 224.

[0056] (Operation and effect of the first embodiment) The information processing system 10 according to this embodiment executes the following steps: an input information acquisition step of acquiring input information to be input to the trained model 222; a version identifier generation step of generating a version identifier for identifying a version of the input information; a storage step of storing the input information and the version identifier corresponding to the input information in a repository; and an update step of generating a new version identifier corresponding to an update of the input information in the repository and storing the updated input information and the corresponding version identifier in the repository. Therefore, a unique version identifier is generated in the version identifier generation step for each piece of input information acquired in the input information acquisition step. This makes it possible to identify different versions of input information, and the change history of each version is clearly managed. Furthermore, the storage step stores the acquired input information and its version identifier in the repository. Since the input information is saved by version, it is possible to distinguish and track past versions from the current version. This history management makes it easy to identify which version of input information was used for the trained model. Furthermore, in the update step, a new version identifier is generated each time the input information in the repository is updated, and this is stored in the repository together with the updated input information. This allows the adjustment history of input information to be managed systematically, enabling efficient changes and adjustments to the input information. Based on this history, it is possible to refer to past changes and quickly generate optimal prompts. Using the input information and its version history managed in this way, it is possible to select the appropriate version of the input information and supply it to the trained model 222. As a result, the desired processing results can be obtained efficiently. Furthermore, comparing and evaluating processing results based on version-managed information promotes the generation and adjustment of optimal prompts. This allows the effectiveness of information processing by the trained model to be maximized.

[0057] Furthermore, the steps executed by the information processing system 10 include an acquisition step in which, when a predetermined version identifier is specified, input information corresponding to the specified version identifier is acquired from the repository, so that by specifying a version identifier desired by a user or another system, input information of the specified version can be accurately acquired from the repository. This makes it possible to easily track and refer to the history and changes of a specific version.

[0058] Furthermore, the information processing system 10 at least compares and evaluates the results of inputting multiple pieces of input information with different version identifiers into the trained model 222. Therefore, by inputting multiple pieces of input information with different version identifiers into the trained model, the performance of each prompt version can be compared and evaluated. This makes it possible to identify which version produces the most accurate results and obtain feedback for improving the accuracy of the model. Furthermore, by comparing multiple input information versions, it is possible to identify the optimal prompt format and standardize that prompt. This makes it possible to efficiently generate optimal prompts and quickly achieve desired processing results. Furthermore, the information processing system 10 can evaluate the diversity of data and the robustness of the model by comparing results using different versions of input information. The operation of the trained model 222 under various conditions can be confirmed, providing clues for improving the fault tolerance and reliability of the entire system. Furthermore, by comparing the histories of multiple versions, it is possible to perform a detailed analysis of how each version of input information affected the processing results. This allows the impact of improvements and changes in each version to be identified and further improvement measures to be taken. Furthermore, comparing results using input information for each version identifier can provide material for tuning and retraining the trained model 222. This improves the performance and ensures stable operation of the trained model 222. Furthermore, using the results of the comparative evaluation, it becomes possible to select and provide the most useful prompt version for the user U. This is expected to optimize the user experience and improve satisfaction. Furthermore, by automating the comparative evaluation between versions, it is possible to efficiently execute a series of processes from prompt generation to evaluation, feedback, and adaptation. This allows the system to always select and utilize the latest, most optimal prompt. This feedback loop allows the quality of the prompts to be continuously improved, and the performance of the trained model 222 to be optimized.

[0059] Furthermore, the storage step in the information processing system 10 clarifies the relationship between input information and output information by linking output information from a trained model based on acquired input information to the input information and storing the linked information in a repository. This makes it easy to track and understand which input information resulted in which output information. Furthermore, since input information and its corresponding output information are stored together, they can be quickly and efficiently referenced and reused as needed. This facilitates new analysis and improvements based on past processing results. Furthermore, storing output information linked to input information in a repository facilitates searches based on specific output results. This makes it possible to quickly obtain input information and its output results that match specific conditions.

[0060] Furthermore, the information processing system 10 generates additional information necessary to obtain output information from the trained model 222 in accordance with the input information and acquires the input information to which the additional information has been added. Therefore, even if the input information is in the form of a template having placeholders, it is possible to determine whether the template can obtain the desired processing results by adding the generated additional information to the placeholders of the template and inputting the template to the trained model 222. In other words, the input information, including templates, can be efficiently evaluated and improved. Based on the evaluation results for each version of the input information, points to improve on the new input information become clear, creating a continuous improvement cycle.

[0061] Furthermore, a version identifier corresponding to the output information is generated and stored in the repository, making each output information uniquely identifiable, thereby making it easy to track which output information was generated for which input information.

[0062] Furthermore, since the method includes an output step of outputting at least one of the input information and the output information in a viewable manner, it becomes possible to compare and evaluate the output results of the trained model for the input information, thereby selecting the optimal prompt and improving the response accuracy of the trained model and the entire system.

[0063] Furthermore, by storing branch information corresponding to input information in the repository, it is possible to evaluate multiple different revisions based on each version identifier and to deploy input information based on various scenarios, thereby achieving desired processing results more efficiently.

[0064] Furthermore, since the method includes a search step in which the input information is searched using search condition information in natural language, the user U can input their own questions or requests in the original format, allowing for stress-free searching, thereby improving the satisfaction of the user U.

[0065] Note that in order to search the input information using search condition information in natural language, pre-structured metadata may be added to the input information. As a specific example, context information such as "intention," "theme," and "classification" may be added to the input information as metadata manually by a user U or automatically by the trained model 222, and then vectorized. Then, the search condition information input in the search step is also vectorized to obtain input information with a high degree of similarity between vectors. This can further improve search accuracy.

[0066] Second Embodiment Next, an information processing system 500 according to a second embodiment of the present invention will be described with reference to Figures 5 and 6. The information processing system 500 according to the second embodiment has the same basic configuration as the first embodiment, and is characterized in that tag information in input information is acquired and stored in a storage unit. Note that components that are the same as those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted.

[0067] (Functional configuration) The functional configuration of the server 70 in the information processing system 500 will be described. FIG. 5 is a diagram showing an example of the functional configuration of the server 70. When executing various programs, the server 70 realizes various functions using hardware resources similar to those of the server 12 of the first embodiment. The server 70 has a communication unit 20, a storage unit 72, and a control unit 74 as the functional configuration realized by the server 70. Each functional configuration is realized when the processor 120 reads and executes a program 720 stored in the memory 122 or the storage 124.

[0068] The storage unit 72 is realized by the memory 122 and the storage 124. The storage unit 72 stores a program 720, a trained model 222, a prompt DB 724, a user information DB 226, and a processing result DB 228.

[0069] The prompt DB 724 stores information indicating prompts input by the user U, templates, and version identifiers, similar to the prompt DB 224 of the first embodiment. The prompt DB 724 also stores tag information, which will be described later.

[0070] The control unit 74 is realized by the processor 120 reading and executing the program 720 from the memory 122 (see FIG. 2) and working in cooperation with other hardware components. The control unit 74 includes an information acquisition unit 742, an information processing unit 744, and an output unit 246.

[0071] Like the information acquisition unit 242 of the first embodiment, the information acquisition unit 742 acquires various types of information required for processing by the information processing unit 744 and various types of information for searching for prompts. The information acquisition unit 742 also acquires tag information. The tag information acquired by the information acquisition unit 742 is tag information directly input by the user U. For example, if the input information is related to technology, the information acquisition unit 742 acquires tag information directly input by the user U, such as "technology." The format in which the user U directly inputs tag information includes direct input of keywords, or input of predefined information by selection. The information acquisition unit 742 also acquires tag information generated by the information processing unit 744 (details of which will be described later) in addition to the tag information directly input by the user U. In other words, the information acquisition unit 742 corresponds to the "input information acquisition unit" and "tag information acquisition unit" of claim 16.

[0072] Similar to the information processing unit 244 of the first embodiment, the information processing unit 744 processes information using the trained model 222 based on prompts, constantly monitors acquired prompts, generates additional information corresponding to placeholders in the prompt template, compares and evaluates output results based on different versions of prompts, and searches for prompts in the storage unit. The information processing unit 744 also generates tag information and stores it in the storage unit. The tag information generated by the information processing unit 744 includes information generated by extracting keywords from input information, information generated using the trained model 222, and information generated using external tagging services via API integration, etc. Specific examples of tag information include the following categories: First, tags based on the subject or theme of the input information include "technology," "business," "health," "education," and "entertainment." Furthermore, tags reflecting the user U's intentions and purposes in the input information include "question," "suggestion," "problem solving," "information search," and "feedback." Second, tags based on the emotion or tone of the input information include "positive," "negative," "neutral," "question," and "gratitude." Tags based on the specific use of the input information include "marketing," "product reviews," "customer support," "educational materials," and "project management." Tags based on specific keywords or phrases contained in the input information include "machine learning," "API," "healthcare," "blockchain," and "leadership." Tags based on the target user demographic or market include "beginners," "experts," "students," "businesses," and "general public." Tags related to specific regions or locations include "Japan," "America," "Tokyo," "New York," and "local information." Tags related to specific times or periods include "2023," "this week," "next month," "annual events," and "history." These tags are generated and managed appropriately based on the content and purpose of the input information. Specifically, consider the example of "Please explain the latest AI technology related to technology."For input information such as "technology," "AI," "education," and "for experts" are generated as tags. In this way, using tag information makes it easier to search and classify input information, improving the efficiency of the information processing system. In other words, the information processing unit 744 corresponds to the "storage execution unit" recited in claim 16.

[0073] (Processing performed by information processing system 500) Next, the operation of the information processing system 500 will be described. Fig. 6 is a flowchart showing an example of the flow of processing by the information processing system 500. The processor 120 reads out the program 720 stored in the storage 124, expands it in the memory 122, and executes it, thereby performing processing. Note that the same processes as those in the first embodiment are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0074] If the input information is new in step S102 (step S102: NO), the processor 120 determines whether or not tag information has been input by the user U through the user terminal 14 (step S200). If the input information has been input (step S200: YES), the processor 120 proceeds to step S104, where the processor 120 stores the tag information in the storage unit in step S106. On the other hand, if the input information has not been input (step S200: NO), the processor 120 generates tag information based on the input information (step S202), and then proceeds to step S104, where the processor 120 stores the tag information in the storage unit in step S106. The processing in step S202 corresponds to the "tag generation step" recited in claim 10.

[0075] (Operation and effect of the second embodiment) The information processing system 500 according to this embodiment has the same configuration as the first embodiment, except that tag information in input information is acquired and stored in a storage unit, and thus provides the same operational effects as the first embodiment. Furthermore, the information processing system 500 includes an input information acquisition step for acquiring input information to be input to a trained model and tag information in the input information, and a storage step for storing the input information and tag information in a repository. Therefore, by using the tag information stored in the repository, input information matching specific conditions can be quickly searched and filtered. This allows for efficient acquisition of necessary data, enabling smooth analysis and processing. Furthermore, by utilizing tag information, users can set appropriate tags based on their own input information and easily search and reference information later. This improves the ease of use of the system and enhances the user experience.

[0076] Furthermore, the information processing system 500 includes a tag generation step that generates tag information based on input information, significantly reducing manual tagging work and enabling efficient data processing. This reduces the burden on users U and administrators, and improves the operational efficiency of the entire system. Furthermore, automatic tag generation enables consistent tagging. This unifies tagging standards for different users and time periods, improving data consistency and accuracy.

[0077] In the above-described embodiment, the configuration includes a tag generation step that generates tag information based on input information, but the configuration is not limited to this and may be such that tag information is not generated but only tag information input directly by user U or tag information from other systems, etc. is acquired.

[0078] (Third embodiment) Next, an information processing system 600 according to a third embodiment of the present invention will be described with reference to Figures 7 and 8. The information processing system 600 according to the third embodiment has the same basic configuration as the first embodiment, and is characterized in that it acquires feedback information in response to input information. Note that the same components as those in the first embodiment are denoted by the same reference numerals, and their description will be omitted.

[0079] (Functional configuration) The functional configuration of the server 80 in the information processing system 600 will be described. FIG. 7 is a diagram showing an example of the functional configuration of the server 80. When executing various programs, the server 80 realizes various functions using hardware resources similar to those of the server 12 of the first embodiment. The server 80 has a communication unit 20, a storage unit 82, and a control unit 84 as functional components realized by the server 80. Each functional component is realized when the processor 120 reads and executes a program 820 stored in the memory 122 or the storage 124.

[0080] The storage unit 82 is realized by the memory 122 and the storage 124. The storage unit 82 stores a program 820, a trained model 222, a prompt DB 824, a user information DB 226, and a processing result DB 228.

[0081] The prompt DB 824 stores information indicating the prompt, template, and version identifier input by the user U, similar to the prompt DB 224 of the first embodiment. The prompt DB 824 also stores feedback information, which will be described later.

[0082] The control unit 84 is realized by the processor 120 reading and executing the program 720 from the memory 122 (see FIG. 2) and working in cooperation with other hardware components. The control unit 84 includes an information acquisition unit 842, an information processing unit 844, and an output unit 246.

[0083] Like the information acquisition unit 242 in the first embodiment, the information acquisition unit 842 acquires various information required for processing by the information processing unit 844 and various information for searching for prompts. The information acquisition unit 842 also acquires feedback information. In other words, the information acquisition unit 842 corresponds to the "feedback information acquisition unit" of claim 19. The feedback information acquired by the information acquisition unit 842 is feedback information on input information directly input by the user U. For example, it may be a quantitative evaluation of satisfaction or accuracy. Specific examples of this evaluation include evaluation scores ranging from 1 (very dissatisfied) to 5 (very satisfied). Furthermore, the feedback information may also include comments expressing specific opinions or impressions about the input information or its processing results. For example, comments such as "The explanation was easy to understand" or "I would like more detailed information." Next, examples of feedback expressing simple positive or negative responses include a thumbs-up (e.g., like) or a thumbs-down. Additionally, tag information may be added to the feedback, such as "useful," "highly relevant," or "specific." Furthermore, the time the user used the input information or the results may also be included in the feedback information. This allows us to measure the usefulness and engagement of the information. Usage time is recorded, for example, in 5, 10, or 30 minutes. Furthermore, the result selected by user U from multiple provided results can also be used as feedback information. Reports of errors and bugs related to input information and results are also important, and error reports such as "broken link" or "incorrect information" are also included in feedback information. Feedback information also includes logs of the actions taken by user U. An action log includes information such as "clicked a link" or "downloaded." Feedback information also includes a satisfaction index indicating satisfaction with specific items. Examples include satisfaction with "response speed," "accuracy of information," and "ease of use." Feedback information also includes requests for additional information or follow-ups from user U. For example, follow-up requests such as "please tell me the next step" or "I would like similar information" are included.

[0084] Similar to the information processing unit 244 of the first embodiment, the information processing unit 844 processes information using the trained model 222 based on prompts, constantly monitors acquired prompts, generates additional information corresponding to placeholders in the prompt template, compares and evaluates output results based on different versions of the prompt, and searches for prompts in the storage unit. The information processing unit 844 also generates at least one of a correction suggestion and a correction plan for the input information as needed based on feedback information. For example, if the input information includes a prompt such as "Please explain in detail the latest trends in AI technology," and the feedback information includes a comment such as "The processing took a long time. It might be better to reduce the number of specific examples," the correction suggestion generated is "Simplify the detailed explanation and reduce the number of specific examples." This correction suggestion could be "Please give an overview of the latest trends in AI technology."

[0085] Next, an example will be described in which an evaluation score is provided as feedback information. If the input information is a prompt saying "Please explain in detail the latest trends in AI technology," and the feedback information is an evaluation score of 2, it is inferred from the score that user U is dissatisfied, and an example of a correction suggestion is generated: "Simplify the content of the prompt to make it easier to understand." This correction suggestion can be "Please give an overview of the latest trends in AI technology."

[0086] Next, an example is given in which thumbs-up / thumbs-down information is provided as feedback information. When the input information is a prompt such as "Please tell me the basics of marketing strategy," and thumbs-down information is provided as feedback information, it is inferred from the feedback information that user U is dissatisfied, and an example of a correction suggestion is generated: "Make the content of the prompt more specific and add practical elements." This correction suggestion can be, "Please tell me the basics of marketing strategy and some specific examples."

[0087] Next, we will look at an example where tag information is provided as feedback information. When the input information is a prompt such as "Please provide a detailed explanation of the optimization method for deep learning algorithms," and the feedback information is tagged with "difficult," an example of a correction suggestion is generated: "Simplify difficult terms and add easy-to-understand explanations." This correction suggestion can be, "Please provide an easy-to-understand explanation for beginners about the optimization method for deep learning algorithms."

[0088] Next, let us consider an example where the feedback information includes usage time. The input information is a prompt saying, "Please explain why you were unable to resolve this issue." The feedback information is "usage time (stay time) 2 minutes." If this is deemed short in this situation, it is assumed that a more detailed explanation is required, and an example of a correction suggestion is generated: "Add a more detailed explanation to encourage users to stay longer." This correction suggestion could be, "Could you please tell me more about how this issue occurred?"

[0089] Next, an example is given in which a result selection is provided as feedback information. When the input information is a prompt such as "Please explain economic theory," and a simple explanation is selected as feedback information, an example of a revision suggestion is generated: "Simplify the content for beginners." This revision suggestion could be, "Explain the basic theory of economics in a way that is easy for beginners to understand." While the above example is a prompt, the same applies to templates. For example, when a prompt template is "Please provide a detailed explanation of the latest information about XXX (XXX is a placeholder). Please include five specific examples." and the feedback information is "This process took a long time. It might be better to provide fewer specific examples," an example of a revision suggestion is generated: "Reduce the number of specific examples and emphasize the overview." This revision suggestion could be, "Please provide the latest information about XXX. Please include two specific examples." Note that in this embodiment, both a revision suggestion and a revision proposal are generated. However, this is not limiting; at least one of the revision suggestion and the revision proposal may be generated.

[0090] (Processing performed by information processing system 600) Next, the operation of the information processing system 600 will be described. Fig. 7 is a flowchart showing an example of the flow of processing by the information processing system 600. The processor 120 reads out a program 720 stored in the storage 124, expands it in the memory 122, and executes it, thereby performing processing. Note that the same processes as those in the first embodiment are denoted by the same reference numerals, and descriptions thereof will be omitted.

[0091] After processing step S136, the processor 120 determines whether feedback information has been acquired (step S300). If feedback information has not been acquired (step S300: NO), the processor 120 terminates the processing based on the program 820. On the other hand, if feedback information has been acquired (step S300: YES), the processor 120 analyzes the feedback information and determines whether at least one of a correction suggestion and a correction plan for the input information is necessary (step S302). If a correction suggestion or the like is not necessary (step S302: NO), the processor 120 terminates the processing based on the program 820. On the other hand, if a correction suggestion or the like is necessary (step S302: YES), the processor 120 generates at least one of a correction suggestion and a correction plan (step S304) and proceeds to step S300. Note that the above-mentioned step S300 corresponds to a "feedback information acquisition step" recited in claim 11. Furthermore, the above-mentioned step S302 corresponds to a "correction recommendation step" recited in claim 12.

[0092] (Operation and effect of the third embodiment) The information processing system 600 according to this embodiment has the same configuration as the first embodiment, except for the acquisition of feedback information on input information, and therefore provides the same operational effects as the first embodiment. Furthermore, because the information processing system 600 acquires feedback information on input information, it can identify which input information requires the most modification and which input information is most effective. This makes it possible to allocate resources most effectively when updating input information and increase the efficiency of development and improvement.

[0093] Furthermore, the information processing system 600 executes a correction recommendation step of generating at least one of a correction suggestion and a correction plan for the input information based on the feedback information, so that the user U does not need to find and correct problems by himself. This reduces the burden on the user U and allows him to use the system comfortably. Furthermore, the synergistic effect of the feedback information and the correction recommendation step allows the input information to be continuously improved.

[0094] In the above-described embodiment, the feedback information is analyzed to generate at least one of a correction suggestion and a correction plan for the input information as needed. However, this is not limited to this. If the feedback information is analyzed and the feedback information is good, other actions may be taken besides generating a correction suggestion or a correction plan, such as increasing the recommendation level of the input information corresponding to the feedback information or setting it as a standard for evaluation. Furthermore, for input information that has received good feedback information, control may be exercised to output output information corresponding to the input information at a predetermined timing. For example, this predetermined timing may be when input information identical to or similar to the input information that has received good feedback information is input at a different timing. In other words, the output information can be returned without processing the information again using the trained model 222, so that output information that is estimated to be highly satisfying for the user U (because the user U has received good feedback information) can be quickly output while reducing computational costs.

[0095] (Variation 1) Furthermore, if we look at the above-mentioned information processing system from a different perspective, the problem (purpose) that the information processing system of this embodiment aims to solve can also be seen as "efficiently obtaining the desired processing results even when switching trained models."

[0096] If the problem is understood as above, the invention as a means for solving the problem would be, for example, as follows: "An information processing method executed by an information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a version identifier generation step of generating a version identifier for identifying a version of the input information; a storing step of storing the input information and the version identifier corresponding to the input information in a repository; an output information acquisition step of acquiring output information resulting from inputting the input information corresponding to a predetermined version identifier into the trained model and a trained model different from the trained model; an evaluation step of comparing and evaluating the output information; an adjustment step of changing the input information when the result of the evaluation is a predetermined result; "An information processing method including the following."

[0097] According to the above configuration, input information corresponding to a predetermined version identifier is input to different trained models, and the output information is compared and evaluated. Then, an adjustment step is performed in which the input information is changed if the evaluation result in the evaluation step is a predetermined result, i.e., dissimilar. This allows the output information to be changed based on the changed input information. In other words, it is possible to make the output information of one trained model similar to the output information of another trained model. This prevents changes in the output information even when switching trained models, thereby efficiently obtaining desired results. Note that the evaluation of the output information of one trained model and the output information of the other trained model can be performed by inputting the respective output information and a prompt to the large-scale language model to compare and evaluate them. Furthermore, in the adjustment step, the input information can also be changed by inputting a prompt to the large-scale language model to adjust the input information based on the evaluation result.

[0098] (Variation 2) Furthermore, if we look at the above-mentioned information processing system from a different perspective, the problem (purpose) that the information processing system of this embodiment is trying to solve can also be seen as "reflecting the updated content in linked external systems, etc."

[0099] If the problem is understood as above, the invention as a means for solving the problem would be, for example, as follows: "An information processing method executed by an information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a version identifier generation step of generating a version identifier for identifying a version of the input information; a storing step of storing the input information and the version identifier corresponding to the input information in a repository; an updating step of generating a new version identifier in response to an update of the input information in the repository and storing the updated input information and the corresponding version identifier in the repository; and an output step of outputting at least one of the input information and the output information in a usable manner."

[0100] According to the above configuration, since at least one of the input information and the output information is output in a usable manner in the output step, even if the input information is updated in the update step, the updated result can be output. As a result, the updated input information can be used in external systems linked to the system that executes this information processing method, so that changes can be handled immediately. In other words, usability can be improved.

[0101] <Additional Notes> The present embodiment includes the following disclosure.

[0102] (Appendix 1) An information processing method executed by an information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a version identifier generation step of generating a version identifier for identifying a version of the input information; a storing step of storing the input information and the version identifier corresponding to the input information in a repository; an updating step of generating a new version identifier in response to an update of the input information in the repository and storing the updated input information and the corresponding version identifier in the repository; An information processing method including:

[0103] (Appendix 2) and an acquisition step of, when a predetermined version identifier is designated, acquiring the input information corresponding to the version identifier from the repository. 1. The information processing method described in Appendix 1.

[0104] (Appendix 3) At least one of comparing and evaluating results of inputting the plurality of pieces of input information with different version identifiers into the trained model, 1. An information processing method according to claim 1 or 2.

[0105] (Appendix 4) storing branch information corresponding to the input information in the repository; An information processing method according to any one of Supplementary Notes 1 to 3.

[0106] (Appendix 5) The storing step associates output information from the trained model based on the acquired input information with the input information and stores the output information in the repository. An information processing method according to any one of Supplementary Notes 1 to 4.

[0107] (Appendix 6) Generate additional information necessary to obtain the output information from the trained model according to the input information, and acquire the input information to which the additional information has been added. 1. The information processing method described in Appendix 5.

[0108] (Appendix 7) generating and storing the version identifier corresponding to the output information in the repository; 1. The information processing method described in Appendix 5.

[0109] (Appendix 8) an output step of outputting at least one of the input information and the output information in a viewable manner, 1. The information processing method described in Appendix 5.

[0110] (Appendix 9) An information processing method executed by an information processing device, an input information acquisition step of acquiring input information to be input to the trained model and tag information in the input information; a storing step of storing the input information and the tag information in a repository; An information processing method including:

[0111] (Appendix 10) a tag generation step of generating the tag information based on the input information. 10. The information processing method according to claim 9.

[0112] (Appendix 11) a search step of searching the input information using search condition information in natural language, An information processing method according to any one of Supplementary Notes 1 to 10.

[0113] (Appendix 12) An information processing method executed by an information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a storing step of storing the input information in a repository; a feedback information acquisition step of acquiring feedback information corresponding to the input information in the repository; An information processing method including:

[0114] (Appendix 13) a correction recommendation step of generating at least one of a correction proposal and a correction plan for the input information based on the feedback information; 13. The information processing method according to claim 12.

[0115] (Appendix 14) In the information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a version identifier generation step of generating a version identifier for identifying a version of the input information; a storing step of storing the input information and the version identifier corresponding to the input information in a repository; an updating step of generating a new version identifier in response to an update of the input information in the repository and storing the updated input information and the corresponding version identifier in the repository; A program for executing an information processing method including the steps of:

[0116] (Appendix 15) In the information processing device, an input information acquisition step for acquiring input information to be input to the trained model; an input information acquisition step of acquiring input information to be input to the trained model and tag information in the input information; a storing step of storing the input information and the tag information in a repository; A program for executing an information processing method including the steps of:

[0117] (Appendix 16) In the information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a storing step of storing the input information in a repository; a feedback information acquisition step of acquiring feedback information corresponding to the input information in the repository; A program for executing an information processing method including the steps of:

[0118] (Appendix 17) An information processing system executed by an information processing device, an input information acquisition unit that acquires input information to be input to the trained model; a version identifier generation unit that generates a version identifier for identifying a version of the input information; a storage execution unit that stores the input information and the version identifier corresponding to the input information in a repository; an update unit that, when the input information in the repository is updated, generates a new version identifier in response to the update and stores the updated input information and the corresponding version identifier in the repository; An information processing system having the above.

[0119] (Appendix 18) An information processing system executed by an information processing device, an input information acquisition unit that acquires input information to be input to the trained model; a tag information acquisition unit that acquires input information to be input to the trained model and tag information in the input information; a storage execution unit that stores the input information and the tag information in a repository; An information processing system having the above.

[0120] (Appendix 19) An information processing system executed by an information processing device, an input information acquisition unit that acquires input information to be input to the trained model; a storage execution unit that stores the input information in a repository; a feedback information acquisition unit that acquires feedback information corresponding to the input information in the repository; An information processing system having the above.

[0121] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. Furthermore, the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0122] 10 Information Processing Systems 12 Server (information processing device) 14 User terminal (information processing device) 220 Programs 222 trained models 242 Information acquisition unit (input information acquisition unit) 244 information processing unit (version identifier generation unit, storage execution unit, update unit) 500 Information Processing Systems 600 Information Processing Systems 742 Information acquisition unit (input information acquisition unit, tag information acquisition unit) 744 Information processing unit (version identifier generation unit, storage execution unit, update unit) 842 Information acquisition unit (input information acquisition unit) 844 Information processing unit (version identifier generation unit, storage execution unit, update unit)

Claims

1. An information processing method executed by an information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a version identifier generation step of generating a version identifier for identifying a version of the input information; a storing step of storing the input information and the version identifier corresponding to the input information in a repository; an updating step of generating a new version identifier in response to an update of the input information in the repository and storing the updated input information and the corresponding version identifier in the repository; An information processing method including:

2. and an acquisition step of, when a predetermined version identifier is designated, acquiring the input information corresponding to the version identifier from the repository. The information processing method according to claim 1 .

3. At least one of comparing and evaluating results of inputting the plurality of pieces of input information with different version identifiers into the trained model, The information processing method according to claim 1 .

4. storing branch information corresponding to the input information in the repository; The information processing method according to claim 1 .

5. The storing step associates output information from the trained model based on the acquired input information with the input information and stores the output information in the repository. The information processing method according to claim 1 .

6. Generate additional information necessary to obtain the output information from the trained model according to the input information, and acquire the input information to which the additional information has been added. The information processing method according to claim 5 .

7. generating and storing the version identifier corresponding to the output information in the repository; The information processing method according to claim 5 .

8. an output step of outputting at least one of the input information and the output information in a viewable manner, The information processing method according to claim 5 .

9. An information processing method executed by an information processing device, an input information acquisition step of acquiring input information to be input to the trained model and tag information in the input information; a storing step of storing the input information and the tag information in a repository; An information processing method including:

10. a tag generation step of generating the tag information based on the input information. The information processing method according to claim 9.

11. a search step of searching the input information using search condition information in natural language, 10. The information processing method according to claim 1 or claim 9.

12. An information processing method executed by an information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a storing step of storing the input information in a repository; a feedback information acquisition step of acquiring feedback information corresponding to the input information in the repository; An information processing method including:

13. a correction recommendation step of generating at least one of a correction proposal and a correction plan for the input information based on the feedback information; The information processing method according to claim 12.

14. In the information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a version identifier generation step of generating a version identifier for identifying a version of the input information; a storing step of storing the input information and the version identifier corresponding to the input information in a repository; an updating step of generating a new version identifier in response to an update of the input information in the repository and storing the updated input information and the corresponding version identifier in the repository; A program for executing an information processing method including the steps of:

15. In the information processing device, an input information acquisition step for acquiring input information to be input to the trained model; an input information acquisition step of acquiring input information to be input to the trained model and tag information in the input information; a storing step of storing the input information and the tag information in a repository; A program for executing an information processing method including the steps of:

16. In the information processing device, an input information acquisition step for acquiring input information to be input to the trained model; a storing step of storing the input information in a repository; a feedback information acquisition step of acquiring feedback information corresponding to the input information in the repository; A program for executing an information processing method including the steps of:

17. An information processing system executed by an information processing device, an input information acquisition unit that acquires input information to be input to the trained model; a version identifier generation unit that generates a version identifier for identifying a version of the input information; a storage execution unit that stores the input information and the version identifier corresponding to the input information in a repository; an update unit that, when the input information in the repository is updated, generates a new version identifier in response to the update and stores the updated input information and the corresponding version identifier in the repository; An information processing system having the above.

18. An information processing system executed by an information processing device, an input information acquisition unit that acquires input information to be input to the trained model; a tag information acquisition unit that acquires input information to be input to the trained model and tag information in the input information; a storage execution unit that stores the input information and the tag information in a repository; An information processing system having the above.

19. An information processing system executed by an information processing device, an input information acquisition unit that acquires input information to be input to the trained model; a storage execution unit that stores the input information in a repository; a feedback information acquisition unit that acquires feedback information corresponding to the input information in the repository; An information processing system having the above.

Images