Information processing system, information processing device, information processing method, and program

Abstract

This system provides an information processing system that improves the accuracy of inference processes using large-scale language models. [Solution] An information processing system is provided that operates based on the inference results of a large-scale language model, comprising: an evaluation unit that evaluates causal relationships between information elements included in the context of the inference process by the large-scale language model; and an inference control unit that controls the inference process to move toward a convergence state that satisfies predetermined conditions based on the evaluated causal relationships.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an information processing system, an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, with the development of generative AI (Artificial Intelligence) technologies such as large language models, an information processing system that generates an inference result based on natural language text input from a user and utilizes the inference result for business support, decision-making support, content generation, etc. has been widely used. In particular, a configuration in which a prompt is sent from a user terminal to a large language model implemented on an external AI server on the cloud and an inference result is obtained as a response thereto has become common.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, since generative AI such as large language models mainly makes inferences based on statistical methods, the generated inference process does not always have logical or causal consistency. For example, the inference proceeds without explicitly evaluating the causal relationship between the information included in the context, and the direction of the inference may deviate from the intention or the result may become unstable.

[0004] The problem to be solved by the present invention is to provide an information processing system that improves the accuracy of the inference process by a large language model.

Means for Solving the Problems

[0005] As one aspect of this disclosure, an information processing system is provided that operates based on the inference results of a large-scale language model, comprising: an evaluation unit that evaluates causal relationships between information elements included in the context of the inference process by the large-scale language model; and an inference control unit that controls the inference process to move toward a convergence state that satisfies predetermined conditions based on the evaluated causal relationships. [Brief explanation of the drawing]

[0006] [Figure 1] This is a schematic diagram showing the overall configuration of the information processing system 1 according to this embodiment. [Figure 2] This is a block diagram showing the functional configuration of device 200. [Figure 3] This is a sequence diagram showing the processing procedure of the information processing system 1 according to this embodiment. [Figure 4] This is a block diagram showing the hardware configuration of computer 90, which is an information processing device. [Figure 5] This figure shows an example of information processing system 1. [Modes for carrying out the invention]

[0007] <1. Embodiments> The information processing system 1 as an embodiment of this designation will be described below with reference to the drawings. In this specification and in each drawing, elements that have already been described are denoted by the same reference numerals, and redundant explanations will be omitted as appropriate. In this embodiment, "part" refers to a concept that includes processing functions realized by hardware resources, or information processing mechanisms that realize such functions in software.

[0008] Furthermore, in this disclosure, "causality" means a relationship in which a change in one information element has a directional influence on a change in another information element or the inference result in the inference process. Here, causality is a functional concept that expresses the direction and intensity of influence in the information processing process in a numerically understandable form. That is, it is defined from the perspective of how much the inference result or intermediate state changes when a certain information element is changed. And, "causal relationship" means the dependency or influence relationship between multiple information elements in the inference process, and is evaluated by numerical indicators such as statistical co-occurrence relationships, contributions based on attention weights, or the amount of change between intermediate representations. Thus, in this embodiment, causal relationship does not mean the philosophical law of causality, but is defined as a concept that mathematically formalizes the direction and dependency structure of information in the inference process. As a result, the causal relationship is treated as an object that can be evaluated, compared, and controlled based on numerical calculations.

[0009] (1.1. Overview of the Information Processing System) As shown in Figure 1, the information processing system 1 according to this embodiment comprises a user terminal 100 operated by the user, a device 200 that performs evaluation and control of the inference process, and an external AI server 300 equipped with a large-scale language model. These are connected to each other via a network N and are configured to collaboratively execute inference processing based on prompts input from the user terminal 100.

[0010] The user terminal 100 is an information processing device that accepts natural language prompt input from the user and receives and displays the inference results from the external AI server 300. The user terminal 100 can be configured as a personal computer, smartphone, tablet, etc. For example, the user terminal 100 sends the prompts received from the user to the external AI server 300. The user terminal 100 also receives the generated inference results via device 200.

[0011] Device 200 is an information processing device having the main functions of this embodiment, receiving inference results from the external AI server 300 and performing evaluation and control of the inference process. In this embodiment, device 200 includes an evaluation unit 211 that evaluates the causal relationships of the context in the inference process, and an inference control unit 212 that controls the inference process to move toward a convergence state that satisfies predetermined conditions based on the evaluation results. Here, predetermined conditions (hereinafter referred to as convergence conditions) mean the conditions that are satisfied when evaluation indicators such as causal vectors, causal depth, consistency with user intent, and output stability in the inference process fall within a preset standard value or acceptable range. This makes it possible to treat the inference process itself as a control target, unlike a configuration that simply receives and outputs inference results, thereby improving the accuracy and stability of the inference. Details of the functions of device 200 will be described later.

[0012] The convergence condition may be determined by a single indicator or by a combination of multiple evaluation indicators. For example, the device 200 may determine that the convergence condition is met if at least one or more of the following conditions are met: the amount of variation in the causal vector is less than or equal to a predetermined tolerance value; the causal depth is within a predetermined range; the similarity to the user's intent is above a predetermined threshold; and the stability of the output (e.g., similarity between regenerations) is above a predetermined threshold. This allows the boundary of the convergence state to be objectively defined by a threshold or tolerance range, ensuring the reproducibility of the convergence determination.

[0013] The external AI server 300 is, for example, an information processing device equipped with a Large Language Model (LLM), which performs inference processing based on the input prompt and generates inference results in text format. For example, the large language model equipped in the external AI server 300 may be a general-purpose generative AI model such as ChatGPT or Gemini, or it may be a dedicated model specialized for a business domain. The large language model may also be a neural network model based on a transformer structure, which forms an inference process by sequentially predicting the next token based on the information contained in the input context. Here, a token means the smallest unit of symbol sequence processed by the large language model, and may include words, parts of words, symbols, or numbers.

[0014] Furthermore, context refers to the set of information that a large-scale language model refers to when predicting the next token. Context may include user prompts, already generated token sequences, externally assigned conditional information, constraint information, and instructional information added by the inference control unit 212 (described later). Context may be represented as a token sequence, or internally as a numerical vector or a representation in the embedding space.

[0015] (1.2. Functional configuration of device 200) Referring to Figure 2, the functional configuration of the device 200 will be described. The device 200 comprises a control unit 210 and a storage unit 220. The control unit 210 has the function of evaluating and controlling the inference process performed by the external AI server 300. The storage unit 220 holds various data used for said evaluation and control.

[0016] The control unit 210 includes an evaluation unit 211 and an inference control unit 212. The evaluation unit 211 has the function of evaluating the causal relationships of the context in the inference process by the large-scale language model using a predetermined method. In this embodiment, the inference process refers to a series of processing steps in which the large-scale language model sequentially predicts tokens based on the input context and generates a sequence of tokens. The inference process may include not only the final output text, but also intermediate information such as the sequence of tokens being generated, the probability distribution at each point in time, the embedding representation, and the attention weight. In this embodiment, the inference process is treated as a sequence of states that changes over time, and this sequence of states is the subject of evaluation and control.

[0017] Furthermore, in this embodiment, "inference process data" means a set of information that can be referenced to evaluate or control the inference process, and may include metadata obtainable from the external AI server 300, or information that can be reconstructed from the inference results on the device 200 side. For example, the generated token sequence, the generation order of each token, the probability or log probability of candidate tokens, generation parameters such as growth and temperature, or intermediate scores may be obtained through an API or log output provided by the external AI server 300. Even if such metadata cannot be obtained from the external AI server 300, the device 200 may construct the inference process data by reestimating state indices (similarity, redundancy, entropy estimate, etc.) described later, based on the generated text sequence, prompts, and external information such as the number of regenerations as inference results.

[0018] Note that internal model information such as attention weights, internal embedding representations, or interlayer activity may not be obtainable depending on the implementation or provision method of the external AI server 300. In this case, the device 200 only needs to perform evaluation processing based on the inference process data that can be obtained. That is, the mode of using internal information is selectively applied only when such information is provided by the external AI server 300, and if it is not provided, inference control may be achieved by evaluation based on external information or reconstruction information.

[0019] As an example, the evaluation unit 211 generates a causal vector as a numerical index indicating the causal directionality of the inference process with respect to the context in the inference process. The causal vector represents, in vector form, the degree of influence of each information element in the context on the inference result, the strength of the dependency relationship, or the feature quantity representing the direction of progress of the inference, and can be calculated based on, for example, the embedded representation of tokens, attention weights, or dependency structure analysis results. This makes it possible to quantitatively grasp in what direction the inference process is developing.

[0020] Specifically, the evaluation unit 211 formulates a causal vector in the inference process by using methods belonging to regression analysis and statistical learning theory. For example, the context at time t is regarded as a set consisting of a plurality of information elements, and each information element is represented as a numerical vector. Also, assuming that the evaluation value regarding the inference result is Y(t), and that the evaluation value can be approximated as a weighted sum of each information element. Then, the weight corresponding to each information element is estimated as a regression coefficient β_i(t), and the vector C(t) obtained by arranging these coefficients is defined as the causal vector. In this way, it is possible to quantitatively represent the degree to which each information element affects the inference result by the magnitude and sign of the coefficient.

[0021] Furthermore, the evaluation unit 211 analyzes the temporal variation of the causal vector by using statistical methods belonging to time series analysis and change point detection. For example, it may be configured to sequentially estimate the regression coefficient using data corresponding to the most recent W inference steps, and update the obtained causal vector C(t) for each time. Then, when the magnitude of the difference between the current causal vector and the previous causal vector exceeds a predetermined threshold, it is determined that the direction of the inference has changed. In this way, by combining sequential estimation based on recent data and change point detection, it is possible to statistically evaluate whether the inference process is stable or rapidly changing direction.

[0022] Further, the evaluation unit 211 may estimate the contribution degree of each information element by using statistical methods belonging to sensitivity analysis and counterfactual virtual estimation. For example, taking the evaluation value in the original context as Y(t), the evaluation value when a certain information element is removed or replaced with another value is calculated. Then, the difference between the two is defined as the contribution degree of that information element, and the vector arranged with these differences may be used as the causal vector C(t). By doing so, it is possible to directly measure how much the inference result changes when each information element does not exist, and it becomes possible to clearly grasp the main factors constituting the inference direction.

[0023] Furthermore, the evaluation unit 211 calculates the causal depth indicating the hierarchy in the causal relationship to which the information included in the context belongs. The causal depth is an index defined based on the number of dependency levels from the premise information to the conclusion in the inference process or the hierarchical structure of the abstraction level, and it quantifies the hierarchical structure of the inference. The evaluation unit 211 evaluates the structural characteristics of the entire inference process by comprehensively considering the causal vector and the causal depth.

[0024] As an example, the evaluation unit 211 formulates the causal depth by using methods belonging to graph theory and hierarchical clustering. For example, each information element included in the context is used as a node, and a directed graph is constructed by representing the dependency relationship or influence relationship between information elements as a directed edge. Then, the dependency path from the node corresponding to the premise information to the node corresponding to the conclusion information is extracted, and the number of levels included in the dependency path is calculated. When there are multiple dependency paths, the number of levels of the path passing through the most levels is defined as the causal depth D(t). By defining it in this way, it is possible to quantify, as an intuitive and discrete number of levels, how many levels of dependency structure the inference has passed through to reach the conclusion.

[0025] Furthermore, the evaluation unit 211 may formulate causal depth using methods belonging to probabilistic graphical models and information theory. For example, the dependency relationships between information elements may be modeled as probabilistic relationships, and the degree to which the uncertainty of other information decreases when certain information is given may be calculated as an indicator. Then, the amount of uncertainty reduction obtained at each stage of inference is sequentially added up, and the cumulative value is defined as the causal depth D(t). With this configuration, causal depth can be expressed not only by simply counting the number of dependency stages, but also as a continuous value indicator that reflects the degree of improvement in the certainty of the information at each stage.

[0026] Furthermore, the evaluation unit 211 may formulate causal depth using methods belonging to time series analysis and statistical model selection theory. For example, the inference process may be approximated as a model in which the current output is determined by the information of multiple past stages, and the number of dependent stages required at that time may be estimated. Specifically, models assuming multiple numbers of dependent stages are compared, the number of stages with the highest goodness of fit is selected based on the information quantity criterion, and that number of stages is defined as the causal depth D(t). In this way, the number of dependent stages required for inference can be objectively evaluated, and the hierarchical complexity of the inference can be quantitatively grasped. Note that each of the above methods may be used mutually exclusive, or multiple methods may be combined to calculate the causal depth. For example, an integrated causal depth can be constructed by combining an index based on the number of dependent stages and an index based on the amount of uncertainty reduction, allowing for flexible design according to the application and accuracy requirements.

[0027] The inference control unit 212 has a function to control the inference process so that it moves toward a convergence state that satisfies predetermined conditions, based on the causal vector and causal depth calculated by the evaluation unit 211. These predetermined conditions include, for example, that the inclusion rate of object phrases extracted from user input is above a predetermined threshold, that the evaluation score indicating logical consistency is above a predetermined value, or that the causal depth does not increase excessively and remains within a stable range. Furthermore, conditions may also be set such that the redundancy of the inference result, the presence or absence of prohibited words, or the output length are within a predetermined range. The inference control unit 212 integrates these multiple conditions as an evaluation function and defines a convergence state as the state in which the value of the evaluation function satisfies a predetermined threshold. Alternatively, the inference control unit 212 may set a judgment function that integrates the index values ​​of each of the above multiple conditions, and determine that the convergence condition has been met when the value of the judgment function satisfies a predetermined threshold.

[0028] The inference control unit 212 defines the inference process as a coordinate system that changes over time. Specifically, the inference state at time t is defined as a state vector S(t), and S(t) is configured as a multidimensional vector whose components include multiple numerical indicators such as (summary value of causal vector, causal depth, entropy of output probability distribution, similarity score with user intent). In this case, S(t) is updated at each token generation step, so the entire inference process is represented as a trajectory moving within the state space. This makes it possible to geometrically grasp the direction of the inference and the tendency towards convergence.

[0029] Furthermore, the inference control unit 212 generates control instructions for the external AI server 300 based on the gradient of the evaluation function or the direction of change of the state vector. For example, if the similarity with the user intent decreases, it adds a re-prompt to emphasize intent-related words, and if the causal depth increases excessively, it instructs summarization or simplification. Also, if the state vector S(t) is changing in a direction that satisfies the convergence condition, it continues generation. In this way, by treating the inference process as a time-series coordinate in the state space and controlling its transitions, the inference result can be gradually guided to a stable state that satisfies predetermined conditions.

[0030] Furthermore, the inference control unit 212 defines a causal convergence point set in accordance with the user's intent and controls the inference process to guide it toward that convergence point. A causal convergence point refers to a state in which the objective conditions and constraints extracted from the user input are satisfied. For example, if the instruction is "Explain with clear justification," the goal is a state in which the proportion of words indicating justification, the frequency of use of logical connecting words, and the number of explanation stages meet predetermined criteria. Also, if the instruction is "State it concisely," the convergence point can be set to a state in which the number of output characters is within the target range and the proportion of redundant expressions is below a predetermined percentage.

[0031] The inference control unit 212 manages the state at each inference step as a set of multiple numerical indicators. For example, indicators such as similarity with user intent, causal depth, the magnitude of the contribution of key information elements, and an indicator showing the uncertainty of the output are used as state indicators. The unit then evaluates how far the current state is from a target reference value and determines that a convergence state has been reached when the difference falls within a predetermined range. By managing the inference state as a combination of numerical indicators in this way, convergence determination can be performed in an objective and reproducible manner.

[0032] Furthermore, the inference control unit 212 sends additional instructions or regeneration requests to the external AI server 300 while referring to the causal vector and causal depth calculated by the evaluation unit 211. For example, if the contribution of information elements unrelated to the user's intent is high in the inference, it instructs the unit to suppress those points. On the other hand, if the contribution of information related to the intent is low, it gives additional instructions to specify the relevant information. Also, if the causal depth is excessively large, it issues instructions to simplify the explanation stage, and if the causal depth is too small, it issues instructions to supplement the prerequisite explanation. In this way, by making stepwise modifications in response to changes in the state indicators, it becomes possible to stably guide the inference process to a result that satisfies predetermined conditions.

[0033] The memory unit 220 stores various data used in processing by the evaluation unit 211 and the inference control unit 212. For example, the memory unit 220 may store an inference process database, a causal vector database, a causal depth database, and a convergence condition database. The inference process database records the generated token sequence, intermediate output, state index values, etc., corresponding to each inference step in chronological order. The causal vector database stores the component values ​​of the causal vector calculated at each time point and their summary statistics. The causal depth database stores depth indicators such as the number of inference stages and uncertainty reduction amount. The convergence condition database stores objective conditions, constraint conditions, threshold information, etc., set according to the application. This data is referenced at each stage of the evaluation process and the inference control process, and is also used as basic information when performing inference again.

[0034] Thus, the device 200 is configured to treat the inference process not merely as an output result, but as a sequence of states subject to evaluation and control. That is, by recording the state at each inference step as a set of numerical indicators and making its transitions traceable, the direction and stability of the inference can be objectively evaluated. The evaluation unit 211 quantifies the structural characteristics of the inference process using causal vectors and causal depth, and the inference control unit 212 makes additional instructions, requests for re-inference, or adjusts constraint conditions based on the evaluation results. Through this iterative coordination of evaluation and control, it becomes possible to gradually guide the inference result to a convergence state that satisfies predetermined conditions, thereby realizing a stable and reproducible output that matches the user's intent.

[0035] (1.3. Processing Procedure) Refer to Figure 3 to explain the processing flow in Information Processing System 1. In step S301, the user terminal 100 receives natural language input from the user and generates a prompt as an inference request. In addition to the input sentence, the user terminal 100 may also accept additional conditions in advance, such as purpose, constraints, output format, reference data, and priority, and formats these together with identification information (user ID, terminal identifier, timestamp, etc.) into a predetermined format. In the formatting process, each element is tagged as structured data and converted into a format that can be mechanically interpreted in subsequent processing. Furthermore, if past dialogue history or related settings exist, these may be added as reference information.

[0036] In step S302, the user terminal 100 sends the prompt generated in step S301 to the device 200. Specifically, the user terminal 100 converts the input statement, additional conditions (purpose, constraints, output format, etc.), identification information, and any added dialogue history information into a data packet according to a predetermined communication protocol and sends it to the device 200. Prior to transmission, the user terminal 100 may perform data format validation, verify the presence or absence of required items, and assign or update a session identifier to format the data into a state that can be uniquely identified in subsequent processing. Furthermore, it may perform retransmission control in the event of a communication error and record a transmission log.

[0037] In step S303, the external AI server 300 performs inference based on the received prompt. The external AI server 300 inputs the received prompt into the large-scale language model and performs inference processing according to the specified acquisition policy (sequential output of generated token sequences, retention of intermediate generation results, assignment of evaluation metadata, live growth control, etc.).

[0038] In step S304, the external AI server 300 transmits the inference results to the device 200. The device 200 receives the inference results and associated metadata transmitted from the external AI server 300. The device 200 stores the received inference results and metadata in the storage unit 220, associating them with a session identifier. Furthermore, the device 200 may verify whether the results conform to a predetermined format and check for missing information or abnormal values.

[0039] The device 200 may store metadata associated with the inference results obtained from the external AI server 300 as inference process data in the storage unit 220. For example, it may be configured to record a sequence of tokens including those in the process of generation, the probability or log probability of each token, the number of generation iterations, and instruction information added by the inference control unit 212 in a time-series relationship. This allows the evaluation unit 211 to calculate a causal vector or causal depth by referring not only to the inference results but also to the progress of the inference.

[0040] In step S305, the evaluation unit 211 of the device 200 performs an evaluation of the inference process based on the received inference result and the inference process data corresponding to the inference result. The evaluation unit 211 generates a causal vector as a numerical index indicating the causal direction of the inference process for the context in the inference process. Furthermore, the evaluation unit 211 calculates the causal depth, which indicates the hierarchy in the causal relationships to which the information contained in the context belongs. By comprehensively considering the causal vector and causal depth, the evaluation unit 211 evaluates the structural characteristics, logical consistency, and direction of inference of the entire inference process and records the evaluation result in the storage unit 220. Furthermore, based on the evaluation result, the evaluation unit 211 determines whether the inference result satisfies predefined convergence conditions. Specifically, it checks whether indicators such as the amount of change in the causal vector, the value of the causal depth, and the degree of consistency with user intent are within a predetermined threshold range, and determines that a convergence state has been reached if these satisfy the convergence conditions, and determines that it has not converged if they do not.

[0041] The inference processing by the external AI server 300 is repeated according to the loop structure shown in Figure 3. Based on the evaluation results by the evaluation unit 211, if it is determined that the inference process does not satisfy the convergence conditions, the inference control unit 212 generates a new control instruction and sends it to the external AI server 300. As a result, the series of processes from step S306 to step S305-2 are repeatedly executed until the convergence conditions are satisfied. On the other hand, if the evaluation unit 211 determines that the convergence conditions are satisfied, step S308 is executed.

[0042] In step S306, the inference control unit 212 of the device 200 performs control to guide the inference process toward a predetermined convergence state based on the causal vector and causal depth calculated by the evaluation unit 211. The inference control unit 212 models the inference process as a state transition that changes along a time series and defines the inference state at time t as the state vector S(t). The inference control unit 212 refers to the transition of the state vector S(t) and the value of the evaluation function and determines the control content so that the state transitions toward a direction in which convergence conditions such as the object phrase coverage rate, logical consistency score, stable range of causal depth, output length, and redundancy are satisfied.

[0043] Specifically, for example, if the similarity to the user's intent decreases, it generates a re-prompt that emphasizes intent-related words, and if the causal depth increases excessively, it instructs summarization or simplification. The inference control unit 212 repeats the inference process by transmitting the control instructions determined in this way to the external AI server 300. The inference control unit 212 then defines a state in which the value of the evaluation function satisfies a predetermined threshold as a convergence state, and controls the entire inference process to guide it step by step towards that convergence state.

[0044] In step S307, the external AI server 300 performs inference again based on the control instructions received from the device 200. The external AI server 300 re-executes the large-scale language model and generates new inference results based on a prompt configuration that reflects additional conditions and regeneration instructions.

[0045] In step S304-2, the external AI server 300 transmits the generated inference results to the device 200. The device 200 receives the inference results, associates them with existing inference process data based on the session identifier, and stores them in the storage unit 220. In step S305-2, the evaluation unit 211 of the device 200 performs a re-evaluation of the inference process based on the regenerated inference results. The evaluation unit 211 recalculates the causal vector and causal depth for the updated context and updates the evaluation content using the recalculated results. Furthermore, the evaluation unit 211 determines, based on the evaluation content, whether the inference results satisfy the predefined convergence conditions.

[0046] In step S308, the device 200 transmits the inference result at the point when the convergence condition is met to the user terminal 100. This allows the device 200 to provide the user terminal 100 with a stable inference result while sequentially evaluating the inference process.

[0047] (1.4. Hardware configuration of the information processing device) Referring to Figure 4, the hardware configuration of the information processing device used as the device 200, etc., will be described. As an example, the information processing device is implemented by the computer 90 shown in Figure 4. The computer 90 may include a CPU 91, ROM 92, RAM 93, storage 94, input interface 95, output interface 96, and communication interface 97.

[0048] The CPU 91 functions as a processor that executes processing. Specifically, the CPU 91 uses the RAM 93 as work memory and the ROM 92 executes programs stored in at least one of the storage 94. During program execution, the CPU 91 controls each component via the system bus 98 and performs various processes. As an example, the CPU 91 functions as a control unit 210.

[0049] ROM92 stores programs that control the operation of computer 90. ROM92 contains the programs necessary for computer 90 to perform each of the processes described above. RAM93 functions as a memory area where the programs stored in ROM92 are loaded.

[0050] The storage 94 stores data necessary for program execution and data obtained through program execution. The storage 94 includes one or more selected from Hard Disk Drives (HDDs) and Solid State Drives (SSDs). As an example, the storage 94 functions as a memory unit 220.

[0051] The input interface (I / F) 95 can connect the computer 90 and the input device 95a. The input interface 95 is, for example, a serial bus interface such as USB. The CPU 91 can read various data from the input device 95a via the input interface 95.

[0052] The output interface (I / F) 96 can connect the computer 90 to the output device 96a. The output interface 96 is a video output interface such as Digital Visual Interface (DVI) or High-Definition Multimedia Interface (HDMI®). The CPU 91 can transmit data to the output device 96a via the output interface 96 and cause the output device 96a to output data.

[0053] The input device 95a is an example of an input means and includes one or more selected from a mouse, keyboard, microphone (voice input), and touchpad. The output device 96a is an example of an output means and includes one or more selected from a display, projector, printer, and speaker. Devices that have the functions of both the input device 95a and the output device 96a, such as a touch panel, may also be used.

[0054] The communication interface (I / F) 97 allows the computer 90 to connect with an external server 97a located outside the computer 90. The communication interface 97 is, for example, a network card such as a LAN card. The CPU 91 can read various data from the external server 97a via the communication interface 97.

[0055] Each process performed by the user terminal 100 or device 200 may be implemented by a single computer 90, or by the cooperation of multiple computers 90.

[0056] The processing of the various data described above may be recorded as a program that can be executed by a computer on a magnetic disk (flexible disk and hard disk, etc.), an optical disk (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, etc.), a semiconductor memory, or another non-transitory computer-readable storage medium.

[0057] For example, information recorded on a recording medium can be read by a computer (or embedded system). The recording format (storage format) of the recording medium is arbitrary. For example, a computer reads a program from the recording medium and, based on this program, causes the processor to execute the instructions written in the program. In a computer, program acquisition (or reading) may be performed via a network.

[0058] (1.5.Summary) As described above, the information processing system 1 is an information processing system that operates based on the inference results of a large-scale language model, and comprises an evaluation unit 211 that evaluates the causal relationships of the context in the inference process by the large-scale language model using a predetermined method, and an inference control unit 212 that controls the inference process to move toward a convergence state that satisfies predetermined conditions based on the evaluated causal relationships. This makes it possible to control the inference process toward satisfying predetermined convergence conditions, correct unnecessary branches and logical leaps based on causal relationships, and improve the accuracy of the inference process by the large-scale language model.

[0059] The evaluation unit 211 and the inference control unit 212 are located outside the external AI server 300 that implements the large-scale language model. This allows for external evaluation and control of the existing model without changing the internal structure or learning parameters of the large-scale language model itself, enabling flexible inference control while utilizing a general-purpose generative AI platform. It should be noted that the term "outside" the external AI server 300, as used to describe the evaluation unit 211 and the inference control unit 212, is not limited to being located in a physically separate enclosure or server, but also includes configurations where they are logically separated and executed as separate processes, separate applications, virtual environments, or containers, even on the same hardware. In other words, as long as they are implemented as a control layer independent of the large-scale language model without changing its internal parameters or learning structure, they qualify as "outside" in this disclosure.

[0060] The evaluation unit 211 evaluates the inference process by generating a causal vector, which is a numerical indicator showing the causal direction of the inference process, for the context of the inference process. This makes it possible to numerically grasp the direction of the inference and the dependencies between pieces of information, and to quantify the characteristics of the inference process and reflect them in the control process.

[0061] The evaluation unit 211 calculates the causal depth, which indicates the hierarchy of causal relationships to which the information contained in the context belongs, and evaluates the inference process taking the causal depth into consideration. This makes it possible to understand what level of abstraction or dependency structure the inference is based on, identify shallow or overly complex inferences, and guide the process toward more structurally stable inferences.

[0062] The inference control unit 212 controls the inference process based on a time-series coordinate sequence model that defines the inference process as coordinates that change over time. This allows the inference process to be represented as a transition in state space, the direction of the transition to be dynamically adjusted according to the evaluation result, and highly convergent inference to be achieved through iterative control processing.

[0063] The inference control unit 212 controls the inference process to guide it toward a causal convergence point set in accordance with the user's intent. This allows the inference to converge to a state that matches the user's desired objective conditions, rather than simply generating a statistically plausible response, and enables the stable provision of an output with a high degree of intent satisfaction.

[0064] Thus, the technical concept in this disclosure does not modify the internal structure of the large-scale language model, but rather is characterized by controlling the causal structure of the input context as an inference request through a causal inference layer placed in the pre-LLM stage that is input to the large-scale language model. In other words, the point of application of the present invention is in the pre-input stage, which allows for the reconstruction of the input conditions themselves before the inference is performed, unlike safety mechanisms within the model, blocking by additional filters, or word-judgment type guardrails based on NG words, etc.

[0065] The main effect (primary effect) of the present invention is to causally evaluate the dependencies and influences between information elements such as objectives, means, targets, and constraints included in the input context, and to invalidate (pre-establishment control) causal structures that would result in a dangerous condition being met, so that they cannot be met before the inference begins. This prevents branching of the inference process towards dangerous conclusions, and while fixing the responsibility boundary (Human-in-the-Command), it improves auditability and explainability by making the preconditions, evaluation indicators, control instructions, and rectification history leading to the inference traceable as logs, thereby contributing to the reduction of risks that could lead to misuse or disputes.

[0066] Furthermore, in this invention, blocking is not based on the presence or absence of a word, but rather on the causal structure determined by causal vectors and causal depth, making it difficult to avoid detection through vocabulary paraphrasing, and enabling control that is consistent with the semantic structure of the input. Moreover, as causal branching is attenuated in the direction where the danger condition is not met, the number of inference iterations and redundant generation are suppressed, and consequently, reductions in the number of tokens, power consumption, and processing costs can be obtained.

[0067] (1.6. Examples) Figure 5 shows the results obtained to verify the effectiveness of the information processing system 1 according to this embodiment. This embodiment is an example for specifically confirming the effects of the present invention, and the present invention is not limited to the measurement conditions, model used, measurement environment, or numerical results. In this embodiment, the operation was verified on an actual iPhone® 12 Pro Max smartphone manufactured by Apple Inc., and the execution time was measured using Time Profiler included in Apple Instruments. The measurement was performed by comparing Case A (configuration without using information processing system 1), which is a comparative example, and Case B (configuration using information processing system 1), which corresponds to the configuration of the present invention, under the same conditions. In both cases, measurements were taken multiple times based on the same input conditions and the same processing content, and the representative value was used for evaluation. The ChatGPT 5.2 model provided by OpenAI was used as the large-scale language model.

[0068] As shown in Figure 5, the execution time measured by the Time Profiler was 61.00 ms in Case A, compared to 2.00 ms in Case B. From these results, it was confirmed that Case B, to which the information processing system 1 was applied, achieved approximately 30.5 times faster processing compared to Case A. This speedup is thought to have been achieved mainly through the cooperation of the causal vector evaluation and causal depth calculation by the evaluation unit 211 and the convergence determination processing by the inference control unit 212. In other words, by quantifying the contribution of each information element in advance through causal vector evaluation, it is possible to suppress regeneration processing based on elements with low contribution and unnecessary branch searches. Furthermore, by controlling the number of inference stages based on the causal depth, it is possible to detect excessively multi-stage development and cyclical inference early and terminate the inference process. In addition, since the regeneration process is stopped when the evaluation function meets a predetermined threshold due to the convergence determination, purposeless iterative calculations are reduced. Thus, it can be considered that selective calculation control based on causal structure directly contributes to shortening the processing time through a reduction in the number of calculation steps.

[0069] Based on the above, in this embodiment, measurements under actual hardware conditions demonstrated that the information processing system 1 can significantly reduce processing time. Thus, the information processing system 1 not only achieves hardware speedup or simple calculation reduction, but the evaluation unit 211 quantifies the causal direction of the inference process as a causal vector, and further grasps the hierarchical complexity of the inference using causal depth. Based on the results, the inference control unit 212 eliminates unnecessary search paths in light of the convergence conditions and allows only the minimum necessary transitions. In other words, the causal vector contributes to directing the search space, the causal depth suppresses the expansion of the search space, and the convergence judgment limits the number of iterations, thereby achieving a reduction in processing time. As a result, while reducing processing time, the structural consistency of the inference process and its alignment with user intent are improved, contributing to improved inference accuracy. Therefore, it can be said that the present invention realizes high-speed and high-precision inference processing through a technical configuration that combines causal evaluation and control.

[0070] <2. Other Embodiments> The information processing system 1 according to this embodiment has been described above, but the application of the technical ideas of this disclosure is not limited to the above embodiment. For example, in the above embodiment, a configuration in which the device 200 includes an evaluation unit 211 and an inference control unit 212 was illustrated, but it is not limited to this. For example, the evaluation unit 211 and the inference control unit 212 may be provided in an information processing device other than the device 200. Specifically, a distributed configuration may be used in which a first device that manages inference execution and a second device that evaluates the inference process and generates control instructions are connected via a network. In this case, evaluation results and state vectors are shared via communication means, and convergence control is realized for the entire system. This facilitates the distribution of processing load, improvement of scalability, and application to cloud environments.

[0071] The specific evaluation methods and control algorithms in the evaluation unit 211 and the inference control unit 212 are not limited to the configurations of causal vectors, causal depth, state vectors, evaluation functions, etc., described above. For example, a policy optimization method based on reinforcement learning, an uncertainty evaluation method based on Bayesian estimation, graph structure analysis, rule-based inference, heuristic evaluation metrics, etc., may be used. Furthermore, the components and weighting methods of the evaluation function can be changed according to the application and purpose, and arbitrary quality indicators such as the stability, explainability, and comprehensiveness of the inference process can be incorporated.

[0072] Furthermore, while the above embodiments demonstrate an example of application to natural language inference using a large-scale language model, the application of the technical concept in this disclosure is not limited to this. For example, it can also be applied to other machine learning models such as image recognition models, speech recognition models, time series prediction models, reinforcement learning models, and graph neural networks. In other words, the technical concept of the present invention can be applied to any configuration that extracts the output process of the model as evaluable features and performs convergence control based on those features. This makes it possible to stabilize and improve the accuracy of inference in a variety of tasks, such as classification problems, regression problems, anomaly detection, and control problems.

[0073] Furthermore, the technical concepts described herein are not limited to software implementation; they may also be implemented as dedicated hardware circuits (ASICs, FPGAs, etc.). A hybrid configuration is also possible, where part of the evaluation process is performed on the terminal side and the remainder on the cloud side. Additionally, an adaptive configuration is possible, where the evaluation history stored in memory is reused as training data, and the evaluation function itself is updated online. Moreover, by combining these with well-known caching technologies, parallel processing technologies, quantization technologies, model compression technologies, etc., processing efficiency and accuracy can be further improved.

[0074] Furthermore, the information processing system 1 may be configured to record the input context before and after rectification, causal vectors, causal depth, evaluation function values, control instructions, number of iterations, convergence judgment results, etc., in a time-series manner. This makes it possible to track which causal elements were rectified and how at a later date, improving explainability and auditability. In addition, the log may be linked with an external audit system or compliance management system.

[0075] Furthermore, the evaluation unit 211 or the inference control unit 212 may be configured to accumulate past rectification history and convergence results as learning data and to sequentially update the weights of the evaluation function or the control policy. For example, it can be configured to learn the optimal control instruction generation strategy using reinforcement learning, or to dynamically adjust the threshold using a statistical optimization method. Such an adaptive configuration enables optimization according to the usage environment and domain.

[0076] While several embodiments of this disclosure have been illustrated above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. Furthermore, the embodiments described above can be implemented in combination with each other. [Explanation of Symbols]

[0077] 1: Information processing system, 90: Computer, 91: CPU, 92: ROM, 93: RAM, 94: Storage, 95: Input interface, 95a: Input device, 96: Output interface, 96a: Output device, 97: Communication interface, 97a: External server, 98: System bus, 100: User terminal, 200: Device, 210: Control unit, 211: Evaluation unit, 212: Inference control unit, 220: Memory unit, 300: External AI server

Claims

1. An information processing system that operates based on the inference results of a large-scale language model, An evaluation unit that evaluates the causal relationships between information elements included in the context during the inference process by the large-scale language model, The system includes an inference control unit that controls the inference process to move toward a convergence state that satisfies predetermined conditions based on the evaluated causal relationship, The evaluation unit is an information processing system that evaluates the inference process by generating a causal vector as a numerical indicator showing the causal direction of the inference process with respect to the context of the inference process.

2. The information processing system according to claim 1, wherein the evaluation unit and the inference control unit are located outside the large-scale language model.

3. The information processing system according to claim 1, wherein the evaluation unit calculates a causal depth indicating the hierarchy in the causal relationships to which the information contained in the context belongs, and evaluates the inference process taking said causal depth into consideration.

4. The information processing system according to claim 1 or 2, wherein the inference control unit controls the inference process based on a time-series coordinate sequence model that defines the inference process as coordinates that change along a time series.

5. The information processing system according to claim 4, wherein the inference control unit controls the inference process so as to guide the inference process toward a causal convergence point set in accordance with the user's intent.

6. An information processing device that operates based on the inference results of a large-scale language model, An evaluation unit that evaluates the causal relationships between information elements included in the context during the inference process by the large-scale language model, The system includes an inference control unit that controls the inference process to move toward a convergence state that satisfies predetermined conditions based on the evaluated causal relationship, The evaluation unit is an information processing device that evaluates the inference process by generating a causal vector as a numerical indicator showing the causal direction of the inference process with respect to the context of the inference process.

7. A method of information processing performed by a computer, The computer performs the steps of evaluating the causal relationships between information elements included in the context of the inference process using a large-scale language model, The computer includes the step of controlling the inference process to move toward a convergence state that satisfies predetermined conditions, based on the evaluated causal relationship. An information processing method in which, in the evaluation step, the computer generates a causal vector as a numerical indicator showing the causal direction of the inference process with respect to the context of the inference process, and evaluates the inference process.

8. On the computer, A step of evaluating the causal relationships between information elements contained in the context during the inference process using a large-scale language model, Based on the evaluated causal relationship, the system performs the step of controlling the inference process to move toward a convergence state that satisfies predetermined conditions. A program that, in the evaluation step, causes the computer to generate a causal vector as a numerical indicator showing the causal direction of the inference process with respect to the context of the inference process, and to evaluate the inference process.

Images