Inference apparatus, inference method and inference procedure

Abstract

One aspect of the present invention relates to an inference apparatus that obtains inference results for each inference model by providing object data to multiple inference models derived from local learning data obtained in different environments to perform prescribed inferences, and by having each inference model perform prescribed inferences. Then, the inference apparatus generates an inference result for the object environment by determining the values ​​of each integration parameter based on the environmental data, weighting the inference results of each inference model using the determined values ​​of each integration parameter, and integrating the weighted inference results.

Description

Technical Field

[0001] This invention relates to inference devices, inference methods, and inference procedures. Background Technology

[0002] In recent years, sensors such as cameras (imaging devices), microphones, temperature sensors, and illuminance sensors have been used in various scenarios to observe certain objects. For example, in scenarios such as product manufacturing production lines, techniques are used to photograph the manufactured products using imaging devices and to check the quality of the products based on the obtained image data.

[0003] The optimal observation method for a sensor may differ depending on the location of the object being observed. For example, when inspecting a product in a bright environment, a faster shutter speed is better, while in a dark environment, a slower shutter speed is better. Furthermore, even within the same location, the optimal observation method may vary depending on the time of day. For instance, the brightness of the location may differ depending on the time of day. Additionally, the brightness of the location may differ even within the same time period, depending on the season. In short, the optimal observation method for a sensor may differ depending on the environment in which it is being observed.

[0004] To properly observe objects, it is desirable to systematically implement sensor observations even in different environments. One example of a systematic approach to sensor observation is the possibility of standardizing the observation methods manually. However, given the diversity of observation environments, it is difficult to standardize the methods uniformly. Furthermore, human subjectivity is introduced during sensor installation, which may prevent the implementation of standardized observation methods. In other words, it is possible that standardized observation methods cannot be achieved in all environments due to human factors.

[0005] Therefore, it is desirable to systematically optimize sensor observation methods in various environments without relying on manual standardization. For example, Patent Document 1 proposes a method for optimizing the detection conditions of alignment marks formed on a substrate. Specifically, a mark detection system is used to detect multiple marks formed on the substrate under multiple illumination and imaging conditions. Next, the detection signal from the mark detection system is analyzed using a prescribed signal processing algorithm, and a decision quantity related to the waveform shape of the detection signal is calculated. Then, the reproducibility of the detection results of multiple marks is evaluated based on the obtained decision quantity, and the multiple illumination and imaging conditions are optimized based on the evaluation results. According to the method of Patent Document 1, illumination and imaging conditions can be optimized in various environments to suit the detection of marks using a prescribed signal processing algorithm.

[0006] In addition, methods for optimizing sensor observation methods using machine learning have been proposed in recent years. For example, Patent Document 2 proposes a classification system that classifies objects into classes based on observation data obtained through repeated object observation actions. This classification system performs reinforcement learning of observation actions based on a reward set according to the amount of entropy reduction derived from the object's classification probability. According to the method in Patent Document 2, the classification system can learn observation actions suitable for object class recognition in various environments through reinforcement learning. Furthermore, for example, Non-Patent Document 1 proposes a method for optimizing the directivity of an ultrasonic array sensor using reinforcement learning.

[0007] Prior art literature

[0008] Patent documents

[0009] Patent Document 1: Japanese Patent Application Publication No. 2012-038794

[0010] Patent Document 2: Japanese Patent Application Publication No. 2012-216158

[0011] Patent Document 3: Japanese Patent Application Publication No. 2011-059924

[0012] Patent Document 4: Japanese Patent Application Publication No. 2017-173874

[0013] Patent Document 5: Japanese Patent Application Publication No. 2018-051664

[0014] Patent Document 6: Japanese Patent Application Publication No. 2019-067238

[0015] Patent Document 7: Japanese Patent Application Publication No. 2019-087096

[0016] Non-patent literature

[0017] Non-Patent Literature 1: Naoki Kotani, Kenji Taniguchi, “Optimal Design Method for Directivity of Ultrasonic Array Sensors Using Reinforcement Learning”, Journal of Systems Control Information Society, 2010, Vol. 23, No. 12, pp. 291-293 Summary of the Invention

[0018] The technical problem that the invention aims to solve

[0019] The inventors of this application have discovered the following problems in the existing optimization methods described above.

[0020] That is, in existing methods, the sensor's observation method is optimized in various environments. Specifically, when optimizing the sensor's observation method through machine learning such as supervised learning, unsupervised learning, and reinforcement learning, learning data is collected in each environment, and a learned machine learning model is constructed using the collected learning data. When the learned machine learning model is applied in the same environment as where the learning data was collected, it can appropriately perform the prescribed inferences (in this case, inferring the optimal observation method for the sensor).

[0021] However, when applied in an environment different from the one where the learning data was collected, the learned machine learning model may not be able to properly execute the prescribed inferences. Therefore, essentially, when optimizing the sensor's observation method in a new environment, learning data is collected in that new environment, and machine learning is then implemented to build the new, fully learned machine learning model using that collected data. Thus, existing methods suffer from the problem of costly optimization of the sensor's observation method in new environments.

[0022] It should be noted that this problem is not unique to scenarios where sensor observation methods are optimized. The same problem arises in all scenarios where a learned machine learning model is used to perform prescribed inferences. These scenarios include, in addition to those involving inferring the optimal sensor observation method, scenarios such as predicting the movement path of a moving object, inferring a suitable conversational strategy for a user, and inferring appropriate action commands for a robot device for a task.

[0023] As specific examples, Reference 3 proposes enabling Hidden Markov Models (HMMs) to predict user movement paths. Reference 4 proposes using deep reinforcement learning to enable learners to determine responses that align with the meeting's objectives. Reference 5 proposes using reinforcement learning to enable learning models to determine the operational conditions of an industrial robot with a robotic arm, including the position, posture, cycle time, torque, and vibration related to objects. Reference 6 proposes using reinforcement learning to enable learning models to determine the control content of the controlled device, which is a robotic device. Reference 7 proposes using reinforcement learning to enable learning models to determine the actions of an autonomous vehicle, which is a robotic device.

[0024] In these scenarios, when used in environments different from the environment in which the learning data was collected, the learned machine learning model may not be able to properly execute the prescribed inferences. Therefore, there may be a cost involved in building a learned machine learning model that can properly execute the prescribed inferences in a new environment.

[0025] Furthermore, the aforementioned problems are not unique to scenarios where machine learning models are built from learned data. For example, they can arise in all scenarios where inference models designed to perform prescribed inferences are derived from learning data using methods other than machine learning, such as manual rule-based methods. Consider, for instance, a scenario where an inference model is generated using manual rule-based methods. Even in such a scenario, when the inference model is applied in an environment different from the environment in which the learning data was collected, it is possible that inputs not considered during generation may be provided, and the inference model may not be able to properly perform the prescribed inferences. Therefore, there may be a cost associated with building an inference model that can properly perform the prescribed inferences in a new environment.

[0026] One aspect of the present invention is made in view of such practical circumstances, and its object is to provide a technique for reducing the cost of constructing inference models that can properly perform prescribed inferences in new environments.

[0027] Technical solutions for solving technical problems

[0028] To solve the above-mentioned technical problems, the present invention adopts the following structure.

[0029] That is, the inference apparatus according to one aspect of the present invention comprises: a first data acquisition unit for acquiring predetermined object data that is the object of a predetermined inference; a second data acquisition unit for acquiring environmental data related to the object environment in which the predetermined inference is performed; a result acquisition unit for providing the object data to multiple inference models derived from local learning data obtained in different environments to perform the predetermined inference, and for performing the predetermined inference on the object data by each inference model, thereby acquiring the inference result of each inference model on the acquired object data; and an integration unit for integrating the inference results according to an integration rule. The inference results of the model are used to generate inference results for the object data in the object environment. The integration rule consists of multiple integration parameters that specify the degree of importance given to the inference results of each inference model in the object environment. The integration of the inference results of each inference model according to the integration rule includes: determining the values ​​of each integration parameter based on the acquired environment data; weighting the inference results of each inference model using the determined values ​​of each integration parameter; and integrating the weighted inference results of each inference model; and an output unit that outputs information related to the generated inference results.

[0030] The inference apparatus described herein utilizes multiple inference models derived from local learning data obtained in different environments to perform prescribed inferences within an object environment. Specifically, the inference apparatus provides each inference model with object data that will become the object of the prescribed inference, enabling each inference model to perform prescribed inferences on the object data, thereby obtaining the inference results of each inference model on the object data. Then, the inference apparatus generates the inference result on the object data in the object environment by integrating the inference results of each inference model according to integration rules. Thus, the inference apparatus described herein does not construct new inference models in order to perform prescribed inferences in the object environment, but rather flexibly applies multiple already constructed inference models.

[0031] The integration rules consist of multiple integration parameters that specify the degree to which the inference results of each inference model are valued in the object-specific context. Generally, it is assumed that the more an inference model is derived from local learning data obtained in a learning environment similar to the object-specific environment, the higher its inference accuracy for object data obtained in the object-specific environment. Conversely, it is assumed that the more an inference model is derived from local learning data obtained in a learning environment dissimilar to the object-specific environment, the lower its inference accuracy for object data obtained in the object-specific environment. Furthermore, sometimes a preferred inference model for the object-specific environment is included among the multiple inference models.

[0032] Therefore, the inference apparatus of this configuration determines the values ​​of each integration parameter of the integration rule based on environmental data related to the object environment in which the prescribed inference is performed. For example, the values ​​of each integration parameter can be determined to prioritize the inference results of at least one of the inference models from learning environments similar to the object environment and high-priority inference models. Then, the inference apparatus of this configuration uses the determined values ​​of each integration parameter to weight the inference results of each inference model and integrates the weighted inference results of each inference model. In this configuration, by adjusting the degree of emphasis on the inference of each inference model according to the object environment in this way, insights (inference models) obtained in various environments can be customized to suit new environments. As a result, the prescribed inference can be performed appropriately in the object environment.

[0033] Therefore, based on this configuration, by utilizing multiple inference models derived from local learning data obtained in different environments, it is possible to appropriately execute the prescribed inferences in the target environment. That is, by flexibly applying the multiple inference models already constructed, it is possible to construct an inference model that can appropriately execute the prescribed inferences in the target environment. Therefore, the cumbersome task of collecting new learning data in the target environment and deriving new inference models based on the collected new learning data can be eliminated. Therefore, based on this configuration, the cost of constructing an inference model that can appropriately execute the prescribed inferences in new environments can be reduced.

[0034] It should be noted that performing a "prescribed inference" refers to performing a certain inference process based on unknown object data. The content of the inference is not particularly limited and can be appropriately determined according to the implementation method. The inference can also be regression or recognition. Prescribed inferences can include, for example, inferring the optimal observation method for sensors, predicting the movement path of a moving object, inferring a suitable conversational strategy for a user, inferring action commands for a robot device suitable for a task, and so on. The representation of the inference result can be appropriately determined according to the content of the inference. The inference result can be represented, for example, by continuous values ​​based on regression, the probability of belonging to a class, etc. Furthermore, the inference result can also be represented by more than one numerical value or more than one class.

[0035] "Object data" refers to any data used in the prescribed inference, and its type is not particularly limited; it can be appropriately selected based on the content of the inference. Object data can be, for example, image data, sound data, numerical data, text data, and other observational data obtained from sensors. Object data can also be referred to as "input data."

[0036] "Environmental data" can be any data related to the environment in which the prescribed inference is performed, and its type is not particularly limited, but can be appropriately selected according to the implementation method. The environment in which the prescribed inference is performed is, for example, the environment in which the object data is obtained. Attributes related to the environment in which the object data is obtained can include all phenomena that may be related to the inference, such as attributes of objects or users that are directly or indirectly associated with the prescribed inference. Similar to object data, environmental data can be, for example, image data, sound data, numerical data, text data, and other observational data obtained by sensors.

[0037] The methods for acquiring object data and environmental data are not particularly limited and can be appropriately selected according to the implementation method. When the object data and environmental data originate from observation data obtained by sensors, they may originate from the same observation data obtained by the same sensor, or from different observation data obtained by different sensors. The object data and environmental data may also include at least partially shared data.

[0038] Each inference model only needs to be able to execute the prescribed inferences, and its structure is not particularly limited, but can be appropriately determined according to the implementation method. Each inference model can be composed of, for example, data tables, functions, rules, etc. The form and content of the outputs (inference results) of the various inference models used in the inference process may not be completely consistent. At least one of the form and content of the outputs of each inference model may differ within the scope of integrating the inference results of the various inference models. For example, multiple inference models for inferring motion commands of robot devices with different numbers of joints can be used to infer motion commands of robot devices in the target environment. The method for integrating the inference results of the various inference models is also not particularly limited, and can be appropriately determined according to the implementation method. For example, the inference results of the various inference models can be integrated by averaging or majority voting.

[0039] Different environments in which local learning data is obtained refer to situations where phenomena such as time, location, object, and user that may influence the given inference are at least partially different. For example, local learning data acquired in different locations can be treated as local learning data acquired in different environments. Furthermore, as another example, local learning data acquired at different times in the same location can be treated as local learning data acquired in different environments. The local learning data used in the derivation of each inference model may also include at least partially identical data.

[0040] The method for deriving each inference model is not particularly limited and can be appropriately selected based on the implementation method. The derivation of each inference model can be done manually or through machine learning. Machine learning methods include, for example, supervised learning, unsupervised learning, and reinforcement learning. The form and type of local learning data are also not particularly limited and can be appropriately determined based on the method for deriving each inference model and the content of the specified inferences.

[0041] In the inference apparatus described above, each inference model can be composed of a fully learned machine learning model that has acquired the ability to perform the prescribed inferences through machine learning using the local learning data. According to this configuration, the cost of building a fully learned machine learning model capable of appropriately performing the prescribed inferences in a new environment can be reduced.

[0042] The inference apparatus described above may also include a learning processing unit that performs machine learning on a computational model for calculating the values ​​of each integration parameter based on the environmental data. This machine learning is configured by training the computational model to make the calculated values ​​of each integration parameter suitable for the prescribed inference in the object environment. Correspondingly, the integration rule may also include a learned computational model constructed through the machine learning. Furthermore, determining the values ​​of each integration parameter based on the environmental data can be achieved by providing the environmental data to the learned computational model and performing computational processing on the learned computational model to obtain the values ​​of each integration parameter as the output of the learned computational model. According to this configuration, by including an integration rule with a learned computational model constructed through machine learning, the prescribed inference can be executed more appropriately in the object environment.

[0043] The number of parameters in an inference model depends on the dimensionality of the object data and the form in which the inference result is represented. Conversely, the number of parameters in a computational model depends on the dimensionality of the environment data and the number of inference models used in the specified inference within the object environment. Therefore, the more complex at least one of the object data and the inference content, the greater the number of parameters in the inference model; conversely, it is possible to suppress the increase in the number of parameters in the computational model.

[0044] For example, consider a scenario where brightness is considered as the object environment, and the optimal observation method for the camera is inferred based on image data obtained from that camera. In this case, the number of parameters in the inference model depends on the number of pixels in the image data and the number of states the camera can take, potentially reaching thousands. On the other hand, the number of parameters in the computational model depends on the dimension of brightness and the number of inference models, but even if dozens of inference models are used in the inference of the optimal observation method, the number would at most be in the tens to hundreds range.

[0045] Therefore, generally speaking, the number of parameters in the computational model should be less than the number of parameters in the inference model. Since the cost of machine learning depends on the number of parameters, the cost of building a learned computational model through machine learning can be kept lower than the cost of building a learned machine learning model as a new inference model. Therefore, in this configuration, the cost of building a learned machine learning model that can properly perform the prescribed inference in a new environment can also be reduced.

[0046] It should be noted that machine learning of the computational model can be performed by an information processing device (computer) other than the inference device. The computational model includes multiple computational parameters used to compute the values ​​of various integration parameters. Machine learning of the computational model can involve adjusting the values ​​of the computational parameters to obtain a desired output suitable for the object environment. That is, training the computational model so that the computed values ​​of the integration parameters are suitable for a given inference in the object environment can involve adjusting the values ​​of the computational parameters of the computational model so that the result calculated based on environmental data, which integrates the inference results of the various inference models using the computed values ​​of the integration parameters, is suitable as an inference result in the object environment.

[0047] In the inference apparatus described above, the machine learning of the computational model can be configured through rewarded reinforcement learning based on the appropriateness of the inference results generated by the integration. Training the computational model may include repeatedly adjusting the values ​​of the computational parameters constituting the computational model to obtain more rewards. The adjustment of the values ​​of the computational parameters may be repeated, for example, until a predetermined condition is met, such as the amount of adjustment of the computational parameter values ​​being below a threshold. According to this configuration, by including the integration rules of the learned computational model constructed through reinforcement learning, the prescribed inferences can be performed more appropriately in the object environment.

[0048] It should be noted that appropriateness refers to the degree of suitability of the inference results generated through integration. For example, in the scenario of inferring the optimal observation method for a sensor, changing the sensor's observation state according to the observation method obtained through integration, the more suitable the observation data obtained by the sensor is for the specified conditions (e.g., image data suitable for product appearance inspection), the more appropriate the inference result of that observation method can be evaluated. Similarly, in the scenario of predicting the movement path of a moving object, the more accurate the predicted path obtained through integration, the more appropriate the predicted path can be evaluated. Furthermore, in the scenario of inferring a conversational strategy suitable for a user, the more suitable the conversational strategy obtained through integration is for the user, the more appropriate the inference result of that conversational strategy can be evaluated. Additionally, in the scenario of inferring action instructions for a robot device suitable for a task, the more suitable the action instructions obtained through integration are for the task, the more appropriate the inference result of that action instruction can be evaluated. Appropriateness can be automatically calculated by a computer based on specified indicators, or it can be specified through operator input. That is, the evaluation of appropriateness can be performed by a computer or by an operator.

[0049] In the inference apparatus described above, determining the values ​​of each integration parameter based on the environmental data can be achieved by acquiring learning environment data related to the environment in which the local learning data is used in deriving the inference models, calculating the fitness of the acquired learning environment data and the environmental data, and determining the values ​​of each integration parameter for each inference model based on the calculated fitness. According to this configuration, the values ​​of each integration parameter can be determined based on the comparison result (i.e., fitness) between the environmental data and the learning environment data, making the more similar the learning environment is to the object environment, the more emphasis is placed on the inference results of the inference model. Therefore, according to this configuration, the cost of constructing an inference model that can appropriately perform prescribed inferences in a new environment can be reduced through a simple method.

[0050] The inference apparatus described above may also include a parameter setting unit that receives a specified value for at least one of the plurality of integrated parameters and sets the value of the at least one integrated parameter to the specified value. Correspondingly, weighting the inference results of each inference model may also include weighting the inference results of the inference model corresponding to the at least one integrated parameter using the set value. According to this configuration, at least a portion of the values ​​of each integrated parameter applied to the inference results of each inference model can be determined by the operator's specification. Thus, the operator's intention (e.g., emphasizing the inference results of a specific inference model) can be reflected in the prescribed inference within the target environment.

[0051] The inference devices described above can be applied to all scenarios in which an inference is performed through an inference model. Scenarios for performing an inference include, for example, inferring the optimal observation method for a sensor, predicting the movement path of a moving object, inferring a conversational strategy suitable for a user, and inferring action commands for a robot device suitable for a task. Alternatively, scenarios for performing an inference could include inferring the attributes of an object photographed in image data obtained from a camera, or inferring features exhibited in observation data obtained from a sensor. Alternatively, scenarios for performing an inference could also include inferring features related to the sensor's observation conditions based on observation data obtained from a sensor. Features related to observation conditions can include all phenomena that may be associated with the observation conditions, such as the distance between the sensor and the observed object, the brightness around the observed object, etc.

[0052] For example, in the inference apparatus described in the above aspect, the prescribed inference may be an inference of features exhibited in the observation data obtained by the sensor based on the observation data. The object data may be the observation data obtained by the sensor. The environmental data may be data related to the environment in which the observation data was acquired. Inferring features exhibited in the observation data obtained by the sensor may be an inference of attributes of the photographed object captured in image data obtained by the camera. According to this configuration, in scenarios where features exhibited in the observation data obtained by the sensor are inferred, the cost of constructing an inference model capable of appropriately performing prescribed inferences in new environments can be reduced. The inference apparatus may be appropriately renamed, for example, an inspection apparatus, a prediction apparatus, a conversation apparatus, a control apparatus, etc., to suit the inference content.

[0053] For example, in the inference apparatus described in the above aspect, the prescribed inference may be a method for inferring a change from the current observation state of an object's properties observed by a sensor to an appropriate observation state for obtaining observation data suitable for prescribed conditions by the sensor. The object data may be data related to the current observation state of the object sensor. The environmental data may be data related to the object observation environment in which the object sensor observes the object's properties. According to this configuration, in scenarios where the properties of an object are observed by a sensor, the cost of constructing an inference model that can appropriately infer the optimal observation method of the sensor in a new environment can be reduced.

[0054] It should be noted that obtaining observation data suitable for the specified conditions corresponds to an appropriate observation state of the sensor. The "specified conditions" can be appropriately specified according to the sensor's observation purpose. For example, in the case of obtaining observation data for product defect detection, the specified conditions can be specified based on performance inferences from the observation data obtained by the sensor, such as whether defect detection can be performed based on the observation data, and whether the accuracy of defect detection meets the benchmark. Product defect detection can be replaced by quality evaluation of the object being photographed (e.g., evaluation of the growth status of crops). Inferring the "change method to an appropriate observation state" can include directly inferring the change method, and indirectly inferring the change method used to guide the observation to an appropriate observation state by inferring an appropriate observation state and calculating the difference between the inferred appropriate observation state and the current observation state.

[0055] In the inference apparatus described above, the environmental data can be obtained from other sensors, different from the object sensor, that observe the object's observation environment. According to this configuration, since environmental data can be easily acquired, a series of processes for integrating the inference results of various inference models can be simplified. Furthermore, according to this configuration, since observation data can be appropriately acquired, the accuracy of the inference results generated through integration can be improved.

[0056] The inference device described above can be connected to an intervention device that changes the observation state of the object sensor by intervening in it. Information related to the inference result may also include instruction information for instructing the intervention device to perform an action to change the observation state of the object sensor according to the inferred change method. The output unit can also cause the intervention device to perform the action to change the observation state of the object sensor according to the inferred change method by sending the instruction information to the intervention device. With this configuration, the observation state of the sensor can be automatically optimized.

[0057] It should be noted that the intervention device only needs to be able to intervene in the observation state of the sensor, and its type is not particularly limited, and can be appropriately selected according to the implementation method. As specific examples, the intervention device can be a pan-tilt unit, an illumination device, etc. A pan-tilt unit can be used to change at least one of the sensor's position and orientation. An illumination device can be used to change the brightness around the object observed by the sensor. The intervention device can have a control device to control its operation. In this case, the inference device can also send command information to the control device to cause the control device to perform the operation of changing the observation state of the object sensor.

[0058] In the inference device described above, the information related to the inference result may also include instruction information for instructing the user to change the observation state of the object sensor according to the change method. The output unit may also output the instruction information to an output device. Based on this configuration, a change method for optimizing the sensor's observation state can be provided to the user. Thus, even users without specialized knowledge can optimize the sensor's observation state according to the object environment.

[0059] In the inference apparatus described in the aforementioned aspect, the sensor may be a camera. The observation data may be image data. The prescribed conditions may be conditions related to capturing the object in the image data to suit the examination of the object's attributes. According to this configuration, in scenarios where the attributes of an object are examined based on image data obtained from a camera, the cost of constructing an inference model capable of appropriately inferring the optimal observation method of the camera in a new environment can be reduced.

[0060] Furthermore, in the inference apparatus described in the aforementioned aspect, the prescribed inference may be a prediction of the movement path of the moving object based on its state. The object data may be data related to the state of the moving object observed by the sensor. The environmental data may be data related to the object observation environment in which the sensor observes the movement of the moving object. According to this configuration, in scenarios where the movement of a moving object is monitored, the cost of constructing an inference model capable of appropriately predicting the movement path of a moving object in a new environment can be reduced. It should be noted that the moving object may be, for example, a living being (human, etc.), a machine (vehicle, etc.), etc.

[0061] Furthermore, in the inference apparatus described above, for example, the prescribed inference may be an inference of a conversational strategy suitable for the user based on the user's conversational actions. The object data may be data related to the conversational actions of the object user. The environment data may be data related to the object conversational environment in which the object user performs conversational actions. According to this configuration, in scenarios where a conversation is generated for a user, the cost of constructing an inference model capable of appropriately inferring a conversational strategy suitable for the user in a new environment can be reduced. It should be noted that the conversational strategy only needs to be related to the determination of the user's conversational actions, and its content is not particularly limited and can be appropriately selected according to the implementation method. The conversational strategy may, for example, specify the content of the conversation, the timing of speaking, the frequency of the conversation, tone of voice, etc.

[0062] Furthermore, for example, in the inference apparatus described in the aforementioned aspect, the prescribed inference may be the inference of action instructions for the robot device suitable for the task based on the state of the robot device. The object data may be data related to the state of the object robot device. The environmental data may be data related to the object task environment in which the object robot device performs the task. According to this configuration, in scenarios involving controlling the actions of a robot device, the cost of constructing an inference model capable of appropriately inferring action instructions for the robot device suitable for the task in a new environment can be reduced.

[0063] It should be noted that the robot device and the task are not particularly limited and can be appropriately selected according to the implementation method. The robot device may be, for example, an industrial robot, a piece of equipment, or an autonomous vehicle. The equipment may be, for example, an air conditioning unit, a lighting unit, etc. When the robot device is an industrial robot, the task may be, for example, placing a workpiece in a target position, etc. When the robot device is an equipment such as an air conditioning unit, the task may be, for example, maintaining a specified temperature, etc., and can be determined according to the type of equipment. When the robot device is an autonomous vehicle, the task may be, for example, moving from the current position to a target position via autonomous driving, etc.

[0064] Furthermore, one aspect of the computational model generation apparatus according to the present invention includes: a data acquisition unit for acquiring learning data used in the machine learning of the computational model, wherein the computational model is used to calculate, based on environmental data, values ​​of multiple integration parameters that specify the degree of importance given to the inference results of each inference model in the target environment, and each inference model is a model derived from local learning data obtained in different environments to perform a specified inference; a learning processing unit for performing the machine learning of the above-mentioned computational model using the acquired learning data, wherein the machine learning is configured by training the computational model so that the calculated values ​​of each integration parameter are suitable for the specified inference in the target environment; and a storage processing unit for storing the results of the machine learning of the computational model in a specified storage area. The computational model generation apparatus may be referred to as a model generation apparatus, a learning apparatus, etc. The learning method of each model (e.g., a machine learning model) is equivalent to the method for generating a learned model (e.g., a learned machine learning model).

[0065] As another aspect of the inference apparatus involved in the above-described embodiments, one aspect of the present invention may be an information processing method implementing each of the above-described inference apparatus or model generation apparatus components, or it may be a program, or it may be a readable storage medium such as a computer storing such a program. A readable storage medium such as a computer refers to a medium that stores information such as a program through electrical, magnetic, optical, mechanical, or chemical action. Furthermore, the inference system according to one aspect of the present invention may be composed of an inference apparatus involved in any of the above-described embodiments and one or more inference model generation apparatuses that derive an inference model based on local learning data. The inference model generation apparatus may be renamed a model generation apparatus, a local learning apparatus, etc. The inference system may also include the above-described computational model generation apparatus.

[0066] For example, one aspect of the inference method relates to an information processing method in which a computer performs the following steps: acquiring specified object data that is the object of a specified inference; acquiring environmental data related to the object environment in which the specified inference is performed; providing the object data to multiple inference models derived from local learning data obtained in different environments to perform the specified inference, and having each inference model perform the specified inference on the object data to acquire inference results of each inference model on the acquired object data; generating an inference result for the object data in the object environment by integrating the inference results of each inference model according to an integration rule, wherein the integration rule consists of multiple integration parameters that specify the degree of importance given to the inference results of each inference model in the object environment, and integrating the inference results of each inference model according to the integration rule includes: determining the values ​​of each integration parameter based on the acquired environmental data; weighting the inference results of each inference model using the determined values ​​of each integration parameter; integrating the weighted inference results of each inference model; and outputting information related to the generated inference result.

[0067] Additionally, for example, one aspect of the inference program relates to a program for causing a computer to perform the following steps: acquiring specified object data that is the object of a specified inference; acquiring environmental data related to the object environment in which the specified inference is performed; providing the object data to multiple inference models derived from local learning data obtained in different environments to perform the specified inference, and causing each inference model to perform the specified inference on the object data, thereby acquiring the inference results of each inference model on the acquired object data; generating an inference result for the object data in the object environment by integrating the inference results of each inference model according to an integration rule, wherein the integration rule consists of multiple integration parameters that specify the degree of importance given to the inference results of each inference model in the object environment, and integrating the inference results of each inference model according to the integration rule includes: determining the values ​​of each integration parameter based on the acquired environmental data, weighting the inference results of each inference model using the determined values ​​of each integration parameter, and integrating the weighted inference results of each inference model; and outputting information related to the generated inference result.

[0068] Invention Effects

[0069] According to the present invention, it is possible to reduce the cost of building a learned machine learning model that can properly perform prescribed inferences in a new environment. Attached Figure Description

[0070] Figure 1AAn example of a scenario in which the present invention is applied is illustrated.

[0071] Figure 1B An example of a scenario in which the present invention is applied is illustrated.

[0072] Figure 2 An example of the hardware configuration of the inference device involved in the implementation is illustrated schematically.

[0073] Figure 3 An example of the hardware configuration of the local learning device involved in the implementation is illustrated schematically.

[0074] Figure 4 An example of the software configuration of the inference device involved in the implementation is illustrated schematically.

[0075] Figure 5A An example of software configuration related to machine learning in a local learning device involved in an implementation is illustrated.

[0076] Figure 5B An example of the software configuration associated with the inference device of the local learning device involved in the implementation is illustrated schematically.

[0077] Figure 6 An example is given of a process related to the generation of a machine learning model after the local learning device involved in the implementation is completed.

[0078] Figure 7A An example of a machine learning model and machine learning method involved in the implementation is shown.

[0079] Figure 7B An example of a machine learning model and machine learning method involved in the implementation is shown.

[0080] Figure 7C An example of a machine learning model and machine learning method involved in the implementation is shown.

[0081] Figure 8 An example is given of the inference processing procedure of the local learning device involved in the implementation.

[0082] Figure 9A An example of a processing procedure related to the inference process of the inference device involved in the implementation is shown.

[0083] Figure 9B An example is given of the processing procedure of a subroutine related to the integration processing of the inference device involved in the implementation.

[0084] Figure 10AAn example of a method for determining the values ​​of the integration parameters involved in the implementation is shown.

[0085] Figure 10B An example is given of the machine learning process of the computational model used in determining the value of the integration parameter involved in the implementation method.

[0086] Figure 10C An example of the computational model and machine learning method involved in the implementation is shown below.

[0087] Figure 10D An example of the computational model and machine learning method involved in the implementation is shown below.

[0088] Figure 11A An example of a method for determining the values ​​of the integration parameters involved in the implementation is shown.

[0089] Figure 11B Examples show the passage Figure 11A This is an example of a process for determining the value of the integration parameters.

[0090] Figure 12A An example of a method for determining the values ​​of the integration parameters involved in the implementation is shown.

[0091] Figure 12B Examples show the passage Figure 12A This is an example of a process for determining the value of the integration parameters.

[0092] Figure 13A The following is an illustrative example of the grouping process of the learned machine learning model involved in the implementation method.

[0093] Figure 13B An example is given of the grouping process performed by the inference device involved in the implementation.

[0094] Figure 13C An example is given of the grouping process performed by the inference device involved in the implementation.

[0095] Figure 14 An example of another scenario in which the invention is applied is illustrated.

[0096] Figure 15 An example of the hardware configuration of an inspection device involved in other methods is illustrated schematically.

[0097] Figure 16A An example illustrating the software configuration of an inspection device involved in other methods is given.

[0098] Figure 16BAn example illustrating the software configuration of an inspection device involved in other methods is given.

[0099] Figure 17 An example of another scenario in which the invention is applied is illustrated.

[0100] Figure 18 An example of the hardware configuration of a prediction device involved in other approaches is illustrated schematically.

[0101] Figure 19A An example illustrating the software configuration of a prediction device involved in other methods is given.

[0102] Figure 19B An example illustrating the software configuration of a prediction device involved in other methods is given.

[0103] Figure 20 An example of another scenario in which the invention is applied is illustrated.

[0104] Figure 21 This example illustrates one instance of the hardware configuration of a conversational device involved in other methods.

[0105] Figure 22A This example illustrates one instance of the software configuration of a conversational device involved in other methods.

[0106] Figure 22B This example illustrates one instance of the software configuration of a conversational device involved in other methods.

[0107] Figure 23 An example of another scenario in which the invention is applied is illustrated.

[0108] Figure 24 An example of the hardware configuration of the control device involved in other methods is illustrated schematically.

[0109] Figure 25A An example illustrating the software configuration of a control device involved in other methods is given.

[0110] Figure 25B An example illustrating the software configuration of a control device involved in other methods is given.

[0111] Figure 26A An example of the hardware configuration of a model generation apparatus involved in other approaches is illustrated schematically.

[0112] Figure 26B An example of the software configuration of a model generation apparatus involved in other methods is illustrated schematically.

[0113] Figure 27An example of the configuration of a local learning device involved in other methods is illustrated schematically.

[0114] Figure 28 An example of another scenario in which the invention is applied is illustrated.

[0115] Figure 29 An outline of the robotic device used in the simulation of the embodiments and comparative examples is shown schematically.

[0116] Figure 30A The results of training the motion of a dynamic robotic device with interpolation conditions through reinforcement learning in the embodiments and comparative examples are shown.

[0117] Figure 30B The results of training the motion of a dynamic robotic device with extrapolation conditions through reinforcement learning in the embodiments and comparative examples are shown. Detailed Implementation

[0118] Hereinafter, an embodiment of the present invention (hereinafter also referred to as "this embodiment") will be described based on the accompanying drawings. However, the embodiments described below are merely illustrative of the present invention in all respects. Of course, various modifications and variations can be made without departing from the scope of the present invention. That is, when implementing the present invention, specific configurations conforming to the embodiments may be appropriately adopted. It should be noted that natural language is used to describe the data in this embodiment; however, more specifically, computer-recognizable analog languages, commands, parameters, machine language, etc., are used to specify them.

[0119] §1 Application Examples

[0120] First, use Figure 1A and Figure 1B An example of a scenario in which the present invention is applied will be described. Figure 1A and Figure 1B An example scenario illustrating the application of this invention is shown below. For example... Figure 1A and Figure 1B As shown, the inference system 100 according to this embodiment includes an inference device 1 and multiple local learning devices 2. The inference device 1 and each local learning device 2 can be interconnected via a network. The type of network can be appropriately selected from, for example, the Internet, wireless communication network, mobile communication network, telephone network, private network, etc.

[0121] Each local learning device 2 involved in this embodiment is a computer configured to perform machine learning using local learning data 30 to execute machine learning model 40. For example... Figure 1AAs shown, each local learning device 2 collects local learning data 30 in different environments and uses the obtained local learning data 30 to perform machine learning on the machine learning model 40. Through this machine learning, each local learning device 2 constructs a fully learned machine learning model 45 that has acquired the ability to perform prescribed inferences.

[0122] The constructed machine learning models 45 that have completed learning are an example of the "inference models derived from local learning data obtained in different environments to perform prescribed inferences" of the present invention. That is, in this embodiment, each inference model is composed of the machine learning models 45 that have completed learning. In the following description, the machine learning models 45 that have completed learning may be referred to as "inference models".

[0123] Machine learning methods can include supervised learning, unsupervised learning, and reinforcement learning. The form and type of the local learning data 30 are not particularly limited; for example, they can be appropriately determined based on the machine learning method and the specified inference content. Each local learning device 2 can be renamed an inference model generation device, a model generation device, etc.

[0124] Performing a prescribed inference involves inferring certain data based on unknown objects. The content of the inference is not particularly limited and can be appropriately determined according to the implementation method. The inference can be regression or recognition. For example, the prescribed inference could be inferring the optimal observation method for a sensor, predicting the movement path of a moving object, inferring a suitable conversational strategy for a user, inferring action commands for a robot device suitable for a task, and so on. Alternatively, the prescribed inference could be inferring the attributes of the photographed object in image data obtained by a camera (imaging device), or inferring features exhibited in the observation data obtained by a sensor. Alternatively, the prescribed inference could be inferring features related to the sensor's observation conditions based on the observation data obtained by the sensor. Features related to observation conditions can include all phenomena that may be associated with the observation conditions, such as the distance between the sensor and the observed object, the brightness around the observed object, etc. The representation of the inference result can be appropriately determined according to the inference content. The inference result can be represented, for example, by continuous values ​​based on regression, the probability of belonging to a class, etc. Furthermore, the inference result can also be represented by more than one numerical value or more than one class.

[0125] The environments in which the local learning data 30 is obtained differ, such as time, location, object, and user, and phenomena that may affect the prescribed inference are at least partially different. For example, local learning data 30 obtained in different locations can be treated as local learning data obtained in different environments. Furthermore, as another example, local learning data 30 obtained in the same location but at different times can be treated as local learning data obtained in different environments. The local learning data 30 used in the construction of each completed machine learning model 45 may include at least partially identical data.

[0126] The environment that receives local learning data 30 and the object environment for performing the prescribed inference by the inference device 1 can be represented by one or more classes or one or more numerical values. When each environment is represented by a numerical value, each environment can also be represented by a numerical range. In this case, the difference between the first environment and the second environment may include, in addition to the first numerical range representing the first environment and the second numerical range representing the second environment not overlapping, the first numerical range and the second numerical range partially overlapping.

[0127] It should be noted that, in Figure 1A and Figure 1B In the example, the inference system 100 has n local learning devices 2_1 to 2_n. Hereinafter, for ease of explanation, further symbols such as _1, _2, and _n will be added to distinguish each device; otherwise, these symbols will be omitted, such as "local learning device 2". Each local learning device 2_1 to 2_n collects local learning data 30_1 to 30_n and performs machine learning on each machine learning model 40_1 to 40_n using the collected local learning data 30_1 to 30_n. Thus, each local learning device 2_1 to 2_n constructs a learned machine learning model 45_1 to 45_n. That is, n learned machine learning models 45_1 to 45_n are generated. However, the number (n) of each local learning device 2 and the generated learned machine learning model 45 is not particularly limited and can be appropriately determined according to the implementation method. Furthermore, the number of learned machine learning models 45 generated by each local learning device 2 is not limited to one and can be two or more.

[0128] On the other hand, the inference device 1 involved in this embodiment is a computer configured to perform prescribed inferences in an object environment using each learned machine learning model 45 derived from local learning data 30 obtained in different environments. For example... Figure 1B As shown, the inference device 1 according to this embodiment acquires specified object data 61, which is the object of the specified inference. Additionally, the inference device 1 according to this embodiment acquires environmental data 63 related to the environment of the object in which the specified inference is performed.

[0129] The object data 61 can be any data used in the prescribed inference, and its type is not particularly limited; it can be appropriately selected according to the content of the inference. Object data 61 can be, for example, image data, sound data, numerical data, text data, or other observational data obtained by sensors. Furthermore, object data 61 can be unprocessed data or processed data (e.g., feature quantities calculated from unprocessed data). Object data 61 can be referred to as "input data".

[0130] Environmental data 63 can be any data related to the object environment in which the prescribed inference is performed, and its type is not particularly limited and can be appropriately selected according to the implementation method. The environment in which the prescribed inference is performed is, for example, the environment in which object data 61 is obtained. Attributes related to the environment in which object data 61 is obtained can include all phenomena that may be related to the inference, such as attributes of objects or users that are directly or indirectly associated with the prescribed inference. Similarly, environmental data 63 can be, for example, image data, sound data, numerical data, text data, or other observation data obtained by sensors. Furthermore, environmental data 63 can also be unprocessed data or processed data.

[0131] The methods for acquiring object data 61 and environmental data 63 are not particularly limited and can be appropriately selected depending on the implementation method. When sensors are used to acquire object data 61 and environmental data 63, object data 61 and environmental data 63 may originate from the same observation data obtained by the same sensor, or they may originate from different observation data obtained by different sensors. In addition, object data 61 and environmental data 63 may also include at least partially common data.

[0132] The inference device 1 of this embodiment provides the acquired object data 61 to each of the learned machine learning models 45 (45_1 to 45_n), enabling each of the learned machine learning models 45 (45_1 to 45_n) to perform a prescribed inference on the object data 61. Thus, the inference device 1 of this embodiment acquires the inference results of each of the learned machine learning models 45 (45_1 to 45_n) on the acquired object data 61. Then, the inference device 1 of this embodiment integrates the inference results of each of the learned machine learning models 45 (45_1 to 45_n) according to integration rule 5 to generate an inference result for the object data 61 in the object environment.

[0133] Specifically, integration rule 5 consists of multiple integration parameters P (P1 to Pn) that specify the degree of importance attached to the inference results of each learned machine learning model 45 (45_1 to 45_n) in the object context. The number of integration parameters can be appropriately determined based on the number of inference models (learned machine learning models 45). More than one integration parameter can be set for each learned machine learning model 45.

[0134] The inference device 1 of this embodiment determines the values ​​of each integrated parameter P (P1 to Pn) based on the acquired environmental data 63. It should be noted that, in determining the values ​​of each integrated parameter P (P1 to Pn), in addition to the environmental data 63, the object data 61 can be further considered. That is, the inference device 1 of this embodiment can also determine the values ​​of each integrated parameter P (P1 to Pn) based on both the object data 61 and the environmental data 63.

[0135] Next, the inference device 1 according to this embodiment uses the determined values ​​of each integration parameter P (P1 to Pn) to weight the inference results of the corresponding learned machine learning models 45 (45_1 to 45_n). Then, the inference device 1 according to this embodiment integrates the weighted inference results of each learned machine learning model 45 (45_1 to 45_n). Thus, the inference device 1 according to this embodiment integrates the inference results of each learned machine learning model 45 (45_1 to 45_n) according to integration rule 5.

[0136] The method for integrating the inference results is not particularly limited and can be appropriately determined according to the implementation method. For example, the inference results of integrating the learned machine learning models 45 can be determined by averaging or majority table. When the specified inference is regression, the integration of inference results can be mainly performed by averaging. In addition, when the specified inference is recognition, the integration of inference results can be mainly performed by majority table. Through this integration process, inference results for object data 61 in the object environment are generated. The inference device 1 according to this embodiment outputs information related to the generated inference results.

[0137] As described above, the inference apparatus 1 according to this embodiment, in order to perform the prescribed inference in the object environment, does not construct a new inference model, but flexibly applies multiple inference models that have already been constructed (learned machine learning models 45_1 to 45_n). Generally speaking, it is envisioned that the more an inference model is derived from local learning data obtained in a learning environment similar to the object environment, the higher the inference accuracy for object data obtained in the object environment. On the other hand, it is envisioned that the more an inference model is derived from local learning data obtained in a learning environment that is not similar to the object environment, the lower the inference accuracy for object data obtained in the object environment. In addition, among the multiple inference models, there may sometimes be an inference model that is recommended to be used preferentially in the object environment.

[0138] Therefore, in the above-described integration process, the inference device 1 of this embodiment determines the values ​​of each integration parameter P (P1 to Pn) of the integration rule 5 based on the environmental data 63 related to the object environment in which the prescribed inference is performed. For example, the values ​​of each integration parameter P (P1 to Pn) can also be determined to prioritize the inference results of at least one of the machine learning models 45 that have completed learning in a learning environment similar to the object environment and the machine learning models 45 with higher priority. In this embodiment, by adjusting the degree of emphasis on the inference of each machine learning model 45 according to the object environment, the insights (machine learning models 45) obtained in various environments can be customized to suit the new environment. As a result, based on the customized insights, in other words, based on the integration result of the weighted inference results of each machine learning model 45, the prescribed inference can be appropriately performed in the object environment.

[0139] Therefore, according to this embodiment, by utilizing multiple fully learned machine learning models 45 derived from local learning data 30 obtained in different environments, the prescribed inferences can be appropriately executed in the target environment. That is, by flexibly applying the multiple fully learned machine learning models 45 that have already been constructed, a new inference model capable of appropriately executing the prescribed inferences in the target environment can be constructed. Therefore, the cumbersome task of collecting new learning data in the target environment and deriving a new inference model (in this embodiment, a fully learned machine learning model) based on the collected new learning data can be eliminated. Therefore, according to this configuration, the cost of constructing an inference model capable of appropriately executing the prescribed inferences in a new environment can be reduced.

[0140] §2 Examples of Composition

[0141] [Hardware Components]

[0142] <Inference apparatus>

[0143] Next, use Figure 2An example of the hardware configuration of the inference device 1 involved in this embodiment will be described. Figure 2 An example of the hardware configuration of the inference device 1 involved in this embodiment is illustrated schematically.

[0144] like Figure 2 As shown, the reasoning device 1 involved in this embodiment is a computer electrically connected to the control unit 11, storage unit 12, communication interface 13, external interface 14, input device 15, output device 16, and driver 17. It should be noted that... Figure 2 In this document, the communication interface and the external interface are recorded as "Communication I / F" and "External I / F".

[0145] The control unit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., which are hardware processors, and is configured to perform information processing based on programs and various data. The storage unit 12 is an example of a memory, such as a hard disk drive or a solid-state drive. In this embodiment, the storage unit 12 stores various information such as the inference program 81, integration rule data 57, learning result data 47, learning data 59, and learning environment data 35.

[0146] Inference procedure 81 is used to enable inference device 1 to perform information processing (described later) related to the prescribed inference in the object environment. Figure 9A and Figure 9B The inference procedure 81 includes a series of commands for this information processing. Integration rule data 57 represents information related to integration rule 5. Each set of learning result data 47 represents information related to each learned machine learning model 45 constructed through machine learning. As described later, the computational model can be used to calculate the values ​​of each integration parameter P (P1 to Pn) based on environment data 63. Integration rule 5 can further include a learned computational model constructed through machine learning. Learning data 59 is used to construct a learned computational model through machine learning. Each set of learning environment data 35 represents information related to the environment in which local learning data 30 is used in the derivation of each inference model (in this embodiment, the machine learning of each learned machine learning model 45). Details will be described later.

[0147] Communication interface 13, such as a wired LAN (Local Area Network) module or a wireless LAN module, is an interface used for wired or wireless communication via a network. It is deduced that by utilizing this communication interface 13, device 1 can perform data communication via a network with other information processing devices (e.g., local learning device 2).

[0148] External interface 14, such as a USB (Universal Serial Bus) port or a dedicated port, is an interface for connecting to external devices. The type and number of external interfaces 14 can be appropriately selected according to the type and number of external devices connected. The inference device 1 can be connected via external interface 14 to a sensor used to acquire at least one of object data 61, environmental data 63, and data that may be associated with them. In addition, the inference device 1 can be connected via external interface 14 to a device that controls an object based on the inference result (e.g., an intervention device, a robotic device, etc., described later).

[0149] Input device 15 is, for example, a mouse, keyboard, or other device for input. Output device 16 is, for example, a display, speaker, or other device for output. The operator can operate the inference device 1 using input device 15 and output device 16.

[0150] The drive 17, such as a CD drive or DVD drive, is a drive device used to read programs stored in the storage medium 91. The type of drive 17 can be appropriately selected according to the type of storage medium 91. At least any one of the above-mentioned inference program 81, integration rule data 57, learning result data 47, learning data 59, and learning environment data 35 can also be stored in the storage medium 91.

[0151] Storage medium 91 is a medium that stores recorded program information through electrical, magnetic, optical, mechanical, or chemical means so that computers and other devices, machines, etc., can read the recorded program information. Inference device 1 can also obtain at least one of the above-mentioned inference program 81, integration rule data 57, learning result data 47, learning data 59, and learning environment data 35 from storage medium 91.

[0152] Here, in Figure 2 In this example, storage medium 91 is illustrated by disc-type storage media such as CDs and DVDs. However, the type of storage medium 91 is not limited to disc-type; it can also be of other types. Examples of storage media other than disc-type storage media include semiconductor memories such as flash memory.

[0153] It should be noted that the specific hardware configuration of the inference device 1 can be appropriately omitted, substituted, or added depending on the implementation method. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be composed of microprocessors, FPGAs (field-programmable gate arrays), DSPs (digital signal processors), etc. The storage unit 12 may also be composed of RAM and ROM included in the control unit 11. At least one of the communication interface 13, external interface 14, input device 15, output device 16, and driver 17 may also be omitted. The inference device 1 may also be composed of multiple computers. In this case, the hardware configuration of each computer may be the same or different. In addition, the inference device 1 may be a general-purpose server device, PC (Personal Computer), etc., in addition to being an information processing device designed specifically for the services provided.

[0154] Local learning device

[0155] Next, use Figure 3 An example of the hardware configuration of each local learning device 2 involved in this embodiment will be described. Figure 3 An example of the hardware configuration of each local learning device 2 involved in this embodiment is shown schematically.

[0156] like Figure 3 As shown, the learning devices 2 involved in this embodiment are computers electrically connected to the control unit 21, storage unit 22, communication interface 23, external interface 24, input device 25, output device 26, and driver 27. It should be noted that... Figure 3 In, with Figure 2 Similarly, the communication interface and external interface are recorded as "Communication I / F" and "External I / F".

[0157] The control units 21 to drivers 27 of each local learning device 2 can be configured in the same way as the control units 11 to drivers 17 of the inference device 1. That is, the control unit 21 includes a CPU, RAM, ROM, etc., as a hardware processor, and is configured to perform various information processing based on programs and data. The storage unit 22 is configured, for example, a hard disk drive, a solid-state drive, etc. The storage unit 22 stores various information such as the learning program 821, the inference program 822, the local learning data 30, the learning result data 47, and the learning environment data 35.

[0158] Learning procedure 821 is used to enable local learning device 2 to perform information processing (described later) related to machine learning of machine learning model 40. Figure 6The learning program 821 includes a series of commands for this information processing. The inference program 822 is a program for causing the local learning device 2 to perform information processing (described later) related to the inferences specified using the learned machine learning model 45. Figure 8 The inference procedure 822 includes a series of commands for this information processing. Local learning data 30 is collected in various environments and utilized in machine learning to enable the machine learning model 40 to perform prescribed inferences. Details will be described later.

[0159] The communication interface 23, such as a wired LAN module or a wireless LAN module, is an interface used for wired or wireless communication via a network. Each local learning device 2 can communicate with other information processing devices (e.g., inference device 1) via the network by utilizing this communication interface 23.

[0160] External interface 24, such as a USB port or a dedicated port, is an interface for connecting to external devices. The type and number of external interfaces 24 can be appropriately selected according to the type and number of external devices connected. Each local learning device 2 can be connected via external interface 24 to a sensor for acquiring at least any one of local learning data 30, learning environment data 35, object data (described later), and data that may be associated with them. In addition, each local learning device 2 can be connected via external interface 24 to a device that controls an object based on inference results (e.g., interventional device, robotic device, described later).

[0161] Input device 25 is, for example, a mouse, keyboard, or other device for input. Output device 26 is, for example, a display, speaker, or other device for output. The operator can operate each local learning device 2 using input device 25 and output device 26.

[0162] The drive 27, such as a CD drive or DVD drive, is a drive device used to read programs stored in the storage medium 92. At least any one of the learning program 821, the inference program 822, and the local learning data 30 described above can also be stored in the storage medium 92. Furthermore, each local learning device 2 can also obtain at least any one of the learning program 821, the inference program 822, and the local learning data 30 from the storage medium 92.

[0163] It should be noted that the specific hardware configuration of each local learning device 2 can be appropriately omitted, substituted, or added depending on the implementation method. For example, the control unit 21 may include multiple hardware processors. The hardware processors may be composed of microprocessors, FPGAs, DSPs, etc. The storage unit 22 may also be composed of RAM and ROM included in the control unit 21. At least one of the communication interface 23, external interface 24, input device 25, output device 26, and driver 27 may also be omitted. Each local learning device 2 may also be composed of multiple computers. In this case, the hardware configuration of each computer may be the same or different. In this embodiment, the hardware configuration of each local learning device 2 is general. However, the relationship between the hardware configurations of each local learning device 2 is not limited to this example. The hardware configuration of one local learning device 2 may also be different from that of other local learning devices 2. In addition, each local learning device 2 may be a general server device, a general PC, etc., in addition to being an information processing device designed specifically for the service provided.

[0164] [Software Composition]

[0165] <Inference apparatus>

[0166] Next, use Figure 4 An example of the software configuration of the inference device 1 involved in this embodiment will be described. Figure 4 An example of the software configuration of the inference device 1 involved in this embodiment is illustrated schematically.

[0167] The control unit 11 of the inference device 1 expands the inference program 81 stored in the storage unit 12 into RAM. Then, the control unit 11 controls each component by interpreting and executing the commands included in the expanded inference program 81 via the CPU. Thus, as... Figure 4 As shown, the inference device 1 according to this embodiment operates as a computer having a first data acquisition unit 111, a second data acquisition unit 112, a result acquisition unit 113, an integration unit 114, an output unit 115, a learning data acquisition unit 116, a learning processing unit 117, a storage processing unit 118, a parameter setting unit 119, and a grouping unit 1110 as software modules. That is, in this embodiment, each software module of the inference device 1 is implemented by the control unit 11 (CPU).

[0168] The first data acquisition unit 111 acquires specified object data 61, which is the object of the specified inference. The second data acquisition unit 112 acquires environmental data 63 related to the object environment in which the specified inference is performed. The result acquisition unit 113 acquires the inference result of each inference model on the acquired object data 61 by providing object data 61 to multiple inference models derived from local learning data 30 obtained in different environments to perform the specified inference on the object data 61, and by causing each inference model to perform the specified inference on the object data 61.

[0169] In this embodiment, each inference model is composed of a machine learning model 45 that has acquired the ability to perform a predetermined inference by using local learning data 30 obtained in different environments. Furthermore, in this embodiment, the result acquisition unit 113 possesses each machine learning model 45 by maintaining each set of learning result data 47. The result acquisition unit 113 sets each machine learning model 45 with reference to each set of learning result data 47. Then, the result acquisition unit 113 provides object data 61 to each machine learning model 45 and acquires the inference result of each machine learning model 45 on the acquired object data 61 by performing a predetermined inference on the object data 61.

[0170] Integration unit 114 integrates the inference results of each inference model according to integration rule 5. Integration rule 5 consists of multiple integration parameters P that specify the degree of importance given to the inference results of each inference model in the object environment. In this embodiment, each integration parameter P (P1 to Pn) specifies the degree of importance given to the inference results of each learned machine learning model 45 (45_1 to 45_n). Integration unit 114 has information related to integration rule 5 by maintaining integration rule data 57. Integration unit 114 sets integration rule 5 with reference to integration rule data 57. Integration unit 114 integrates the inference results of each learned machine learning model 45 (45_1 to 45_n) according to integration rule 5.

[0171] Specifically, the integration unit 114 determines the values ​​of each integration parameter P (P1 to Pn) based on the acquired environmental data 63. It should be noted that, in determining the values ​​of each integration parameter P (P1 to Pn), in addition to the environmental data 63, the object data 61 can also be considered. Next, the integration unit 114 uses the determined values ​​of each integration parameter P to weight the inference results of the corresponding inference models. Then, the integration unit 114 integrates the weighted inference results of each inference model.

[0172] In this embodiment, the integration unit 114 uses the determined values ​​of each integration parameter P (P1 to Pn) to weight the inference results of each learned machine learning model 45 (45_1 to 45_n). Then, the integration unit 114 integrates the weighted inference results of each learned machine learning model 45 (45_1 to 45_n). Thus, the integration unit 114 integrates the inference results of each learned machine learning model 45 (45_1 to 45_n) according to integration rule 5.

[0173] Through the above integration process, the integration unit 114 generates an inference result on the object data 61 in the object environment. The output unit 115 outputs information related to the generated inference result.

[0174] In this embodiment, the computational model can be used to determine the values ​​of each integration parameter P (P1 to Pn) based on the environmental data 63. The learning data acquisition unit 116, the learning processing unit 117, and the storage processing unit 118 perform information processing related to the generation of the learned computational model. Specifically, the learning data acquisition unit 116 acquires the learning data 59 used in the machine learning of the computational model 51, which is used to calculate the values ​​of each integration parameter P (P1 to Pn) based on the environmental data 63. The learning processing unit 117 performs machine learning of the computational model 51 using the acquired learning data 59. Machine learning is configured by training the computational model 51 so that the calculated values ​​of each integration parameter P (P1 to Pn) are suitable for a given inference in the object environment. As a result, the learned computational model 52 is generated. The storage processing unit 118 generates information related to the result of the machine learning of the computational model 51 (i.e., the learned computational model 52) and stores the generated information in a specified storage area. The generated information related to the learned computational model 52 can be included in the integration rule data 57.

[0175] In this case, the integration unit 114 can also provide environmental data 63 (and object data 61) to the learned operational model 52, and obtain the values ​​of each integration parameter P (P1 to Pn) as the output of the learned operational model 52 by performing the operational processing of the learned operational model 52. Thus, the integration unit 114 can also determine the values ​​of each integration parameter P (P1 to Pn) based on the environmental data 63 (and object data 61).

[0176] Furthermore, in this embodiment, the values ​​of at least a portion of the plurality of integration parameters P (P1 to Pn) can be specified by an operator such as a user. The parameter setting unit 119 performs information processing related to the specification of the values ​​of the integration parameters. Specifically, the parameter setting unit 119 receives the specification of the value of at least one of the plurality of integration parameters P (P1 to Pn). Then, the parameter setting unit 119 sets the value of at least one integration parameter to the specified value. Information related to the specified setting can be included in the integration rule data 57. In this case, weighting the inference results of each of the above-mentioned inference models (the learned machine learning model 45) can include weighting the inference results of the inference model (the learned machine learning model 45) corresponding to at least one integration parameter.

[0177] Furthermore, in this embodiment, each inference model (the learned machine learning model 45) can also be grouped. The groups can be appropriately set according to the type of inference, purpose, target environment, etc. The grouping unit 1110 performs information processing related to the grouping of the inference models (the learned machine learning models 45). Specifically, the grouping unit 1110 assigns each inference model (the learned machine learning model 45) to at least one of multiple groups. Correspondingly, the result acquisition unit 113 can also determine the group used in the target environment from the set multiple groups, and perform the processing of acquiring the inference results for the inference models (the learned machine learning models 45) belonging to the determined group. Similarly, the integration unit 114 can also perform the integration processing for the inference results of the inference models (the learned machine learning models 45) belonging to the determined group. Information related to the grouping results of each inference model (the learned machine learning model 45) can be appropriately stored in a designated storage area.

[0178] Local learning device

[0179] (A) Learning Processing

[0180] Next, use Figure 5A An example of the software configuration related to the learning processing of each local learning device 2 involved in this embodiment will be described. Figure 5A An example of the software configuration related to the learning processing of each local learning device 2 involved in this embodiment is shown schematically.

[0181] Each local learning device 2's control unit 21 expands the learning program 821 stored in the storage unit 22 into RAM. Then, the control unit 21, through the CPU, interprets and executes the commands included in the expanded learning program 821 in RAM, controlling each component. Thus, as... Figure 5AAs shown, each local learning device 2 in this embodiment operates as a computer having a learning data acquisition unit 21, an environmental data acquisition unit 212, a learning processing unit 213, and a storage processing unit 214 as software modules. That is, in this embodiment, each software module of each local learning device 2 related to learning processing is implemented by the control unit 21 (CPU).

[0182] The learning data acquisition unit 211 acquires local learning data 30 used in the derivation of the inference model. In this embodiment, the learning data acquisition unit 211 acquires local learning data 30 used in the machine learning of the machine learning model 40. The environment data acquisition unit 212 acquires learning environment data 35 related to the environment in which the local learning data 30 is obtained (hereinafter also referred to as the "learning environment"). The learning environment data 35 is the same type of data as the environment data 63.

[0183] The learning processing unit 213 uses the acquired local learning data 30 to perform machine learning on the machine learning model 40. Through this machine learning, a fully learned machine learning model 45 is constructed (generated) that has acquired the ability to perform the prescribed inferences. The maintenance processing unit 214 generates information related to the constructed fully learned machine learning model 45 as learning result data 47, and saves the generated learning result data 47 in a designated storage area.

[0184] In this embodiment, the collection of local learning data 30 may be performed in different environments among the various local learning devices 2. Then, a learned machine learning model 45 is generated based on the obtained local learning data 30. As a result, multiple learned machine learning models 45 can be obtained, which are derived from the local learning data 30 obtained in different environments to perform prescribed inferences.

[0185] (B) Inference Processing

[0186] Next, use Figure 5B An example of the software configuration related to the inference processing of each local learning device 2 involved in this embodiment will be described. Figure 5B An example of the software configuration related to the inference processing of each local learning device 2 involved in this embodiment is shown schematically.

[0187] Each local learning device 2's control unit 21 expands the inference program 822 stored in the storage unit 22 into RAM. Then, the control unit 21, through the CPU, interprets and executes the commands included in the expanded inference program 822 into RAM, controlling each component. Thus, as... Figure 5BAs shown, each local learning device 2 in this embodiment operates as a computer having an object data acquisition unit 216, an inference unit 217, and an output unit 218 as software modules. That is, in this embodiment, each software module of each local learning device 2 related to inference processing is also implemented by the control unit 21 (CPU).

[0188] The object data acquisition unit 216 acquires object data 225, which is the object of a predetermined inference. Object data 225 is the same type of data as the object data 61 obtained by the inference device 1. The inference unit 217 includes an inference model derived from the local learning data 30. The inference unit 217 performs a predetermined inference on the object data 225 using the inference model. In this embodiment, the inference unit 217 includes a learned machine learning model 45 by maintaining the learning result data 47. The inference unit 217 sets the learned machine learning model 45 with reference to the learning result data 47. Then, the inference unit 217 provides the object data 225 to the learned machine learning model 45 and performs computational processing on the learned machine learning model 45. Thus, the inference unit 217 acquires the result of performing the predetermined inference on the object data 225 as the output of the learned machine learning model 45. The output unit 218 outputs information related to the inference result on the object data 225.

[0189] <Other>

[0190] The software modules of inference device 1 and each local learning device 2 will be described in detail in the operational examples described later. It should be noted that in this embodiment, an example has been described where each software module of inference device 1 and each local learning device 2 is implemented using a general-purpose CPU. However, some or all of the above software modules may also be implemented using one or more dedicated processors. Furthermore, regarding the software configuration of inference device 1 and each local learning device 2, software modules may be omitted, replaced, or added as appropriate depending on the implementation method.

[0191] §3 Action Examples

[0192] [Local learning device]

[0193] (A) Learning Processing

[0194] Next, use Figure 6 Examples of actions of each local learning device 2 related to the generation of the inference model are explained. Figure 6This is a flowchart illustrating an example of the processing related to the generation of the machine learning model 45 after learning by each local learning device 2. However, the processing described below is merely an example, and each step can be modified as much as possible. Furthermore, for each processing described below, steps can be omitted, substituted, or added appropriately according to the implementation method.

[0195] (Steps S101 and S102)

[0196] In step S101, the control unit 21 operates as a learning data acquisition unit 211 to acquire the local learning data 30 used in the derivation of the inference model. In this embodiment, the control unit 21 acquires the local learning data 30 used in the machine learning of the machine learning model 40.

[0197] In step S102, the control unit 21 operates as a learning processing unit 213, performing machine learning on the machine learning model 40 using the acquired local learning data 30. Through this machine learning, the control unit 21 is able to generate a fully learned machine learning model 45 that has acquired the ability to perform predetermined inferences.

[0198] The composition of the machine learning model 40 and the machine learning method are not particularly limited and can be appropriately determined according to the implementation method. Machine learning methods, such as supervised learning, unsupervised learning, and reinforcement learning, can be used. The machine learning model 40 can be represented, for example, by a data table or a function. When the machine learning model 40 is represented by a function, it can be composed of, for example, a neural network, a linear function, or a decision tree. The type of neural network is not particularly limited and can be appropriately determined according to the implementation method. The neural network constituting the machine learning model 40 can be, for example, a fully connected neural network, a convolutional neural network, a recurrent neural network, or a combination thereof. The following will illustrate three examples of the composition of the machine learning model 40 and the machine learning method.

[0199] 1.The first case

[0200] Figure 7A The diagram schematically illustrates the structure of a machine learning model 40 and a first example of a machine learning method. In this first example, the machine learning model 40 employs a neural network, and the machine learning method uses supervised learning. It should be noted that... Figure 7A For ease of explanation, an example of each of the local learning data 30, the machine learning model 40, and the machine learning model 45 after learning will be described as local learning data 301, machine learning model 401, and machine learning model 451 after learning.

[0201] 1-1. Example of a Machine Learning Model

[0202] In the first example, the machine learning model 401 consists of a four-layer fully connected neural network. Specifically, starting from the input side, the machine learning model 401 sequentially includes an input layer 4011, intermediate (hidden) layers (4012, 4013), and an output layer 4014. However, the structure of the machine learning model 401 is not limited to this example and can be appropriately determined according to the implementation method. For example, the number of intermediate layers in the machine learning model 401 is not limited to two; it can be one or less, or more than three. Intermediate layers can also be omitted. In addition, the machine learning model 401 can also include other types of layers such as convolutional layers, pooling layers, and recursive layers. For example, long short-term memory can be used for recursive layers.

[0203] The number of neurons (nodes) included in each layer 4011-4014 is not particularly limited and can be appropriately selected according to the implementation method. Neurons in adjacent layers are appropriately connected, and weights (connection weights) are assigned to each connection. Figure 7A In the example, the neurons in layers 4011-4014 are connected to all neurons in adjacent layers. However, the connection relationships of neurons are not limited to this example and can be appropriately determined according to the implementation method. Each neuron is given a threshold, and the output of each neuron is basically determined based on whether the sum of the products of each input and each weight exceeds the threshold. The weights of the connections between the neurons in layers 4011-4014 and the thresholds of each neuron are examples of computational parameters that are adjusted through machine learning.

[0204] 1-2. Examples of Local Learning Data Composition

[0205] The local learning data 301 used in the supervised learning of this machine learning model 401 consists of multiple learning datasets 310, each including a combination of training data 311 (input data) and positive response data 312 (supervision signal). Training data 311 is the data that becomes the object of the prescribed inference, and is the same type of data as the object data (61, 225). Positive response data 312 represents the result (positive response) of performing the prescribed inference on the training data 311. Positive response data 312 can also be called labels.

[0206] The content of training data 311 and forward data 312 can be appropriately selected based on the inferences that enable the machine learning model 401 to master. For example, if the machine learning model 401 is to master the ability to infer the optimal observation method of the sensor, training data 311 can consist of data related to the current observation state of the sensor, and forward data 312 can consist of data representing the method of changing from the current observation state to an appropriate observation state. Similarly, if the machine learning model 401 is to master the ability to predict the movement path of a mobile object, training data 311 can consist of data related to the state of the mobile object, and forward data 312 can consist of data representing the actual or virtual movement path of the mobile object in that state. Furthermore, if the machine learning model 401 is to master the ability to infer a conversational strategy suitable for a user, training data 311 can consist of data related to the user's conversational actions, and forward data 312 can consist of data representing a conversational strategy suitable for that user. Furthermore, for example, in the case where the machine learning model 401 acquires the ability to infer action instructions for a robot device suitable for a task, the training data 311 can consist of data related to the state of the robot device, and the forward data 312 can consist of data representing action instructions suitable for completing the task. Additionally, for example, in the case where the machine learning model 401 acquires the ability to infer features expressed in observation data obtained from sensors, the training data 311 can consist of observation data obtained from sensors, and the forward data 312 can consist of data representing features expressed in that observation data.

[0207] 1-3. Regarding step S101

[0208] In step S101 described above, the control unit 21 acquires local learning data 301 composed of multiple learning datasets 310. The method for generating each learning dataset 310 is not particularly limited and can be appropriately selected depending on the implementation method. For example, various situations for performing a predetermined inference can be created, either physically or virtually, and data of the same type as the object data (61, 225) can be acquired as training data 311 in these created situations. As a specific example, if a sensor is used in acquiring the object data (61, 225), a sensor of the same type as the sensor used is prepared, and the various situations for performing the predetermined inference are observed using the prepared sensor, thereby acquiring training data 311. Next, the predetermined inference is performed on the acquired training data 311. In this scenario, the predetermined inference can be performed manually by an operator or the like. Then, the result (correct solution) of performing the predetermined inference on the training data 311 is associated with the training data 311. Thus, each learning dataset 310 can be generated.

[0209] Each learning dataset 310 can be automatically generated by computer operations or manually generated by an operator. Furthermore, the generation of each learning dataset 310 can be performed by each local learning device 2 or by a computer other than the local learning devices 2. When each learning dataset 310 is generated by each local learning device 2, the control unit 21 automatically or manually executes the aforementioned series of generation processes via the operator's operation through the input device 25, thereby acquiring local learning data 301 composed of multiple learning datasets 310. On the other hand, when each learning dataset 310 is generated by another computer, the control unit 21 acquires local learning data 301 composed of multiple learning datasets 310 generated by the other computer, for example, via a network, storage medium 92, etc. It is also possible for each local learning device 2 to generate a portion of the learning datasets 310, and for one or more other computers to generate other learning datasets 310. The number of acquired learning datasets 310 is not particularly limited and can be appropriately selected according to the implementation method.

[0210] 1-4. Regarding step S102

[0211] In step S102 above, the control unit 21 performs supervised learning (machine learning) on ​​the machine learning model 401 using the acquired local learning data 301. In the first example, the control unit 21 trains the machine learning model 401 through machine learning, such that for each learning dataset 310, when the training data 311 is input to the input layer 4011, the output layer 4014 outputs an output value suitable for the correct solution data 312. Thus, the control unit 21 generates a fully trained machine learning model 451 that has acquired the ability to perform predetermined inferences.

[0212] As an example of a specific process in supervised learning, the control unit 21 first prepares a machine learning model 401, which will be the object of machine learning processing. The structure of the prepared machine learning model 401 (e.g., the number of layers, the number of neurons in each layer, the connection relationships between neurons in adjacent layers, etc.) and the initial values ​​of each operational parameter (e.g., the initial values ​​of the weights of the connections between neurons, the initial values ​​of the thresholds of each neuron, etc.) can be provided through a template or through operator input. In addition, in the case of relearning, the control unit 21 can also prepare the machine learning model 401 based on the learning result data obtained through past supervised learning.

[0213] Next, the control unit 21 uses the training data 311 included in each learning dataset 310 as input data and the forward pass data 312 as supervision signals to perform learning processing on the neural network constituting the machine learning model 401. This learning processing can use batch gradient descent, stochastic gradient descent, mini-batch gradient descent, etc.

[0214] For example, in the first step, the control unit 21 inputs the training data 311 into the machine learning model 401 for each learning dataset 310 and executes the computational processing of the machine learning model 401. That is, the control unit 21 inputs the training data 311 into the input layer 4011 and sequentially performs firing determination on each neuron included in each layer 4011 to 4014, starting from the input side. Through this computational processing, the control unit 21 obtains from the output layer 4014 the output value corresponding to the result of the machine learning model 401 in training performing a prescribed inference on the training data 311.

[0215] In the second step, the control unit 21 calculates the error between the output value obtained from the output layer 4014 and the positive solution data 312 based on the loss function. The loss function is a function that evaluates the difference between the output of the machine learning model 401 and the positive solution. The larger the difference between the output value obtained from the output layer 4014 and the positive solution data 312, the larger the error calculated by the loss function. The type of loss function used in the calculation of this error is not particularly limited and can be appropriately selected according to the implementation method.

[0216] In the third step, the control unit 21 uses the gradient of the calculated output value error to calculate the errors in the values ​​of various operational parameters of the machine learning model 401, such as the weights of connections between neurons and the thresholds of each neuron, through back propagation. In the fourth step, the control unit 21 updates the values ​​of each operational parameter of the machine learning model 401 based on the calculated errors. The degree of updating the operational parameter values ​​can be adjusted by the learning rate.

[0217] The control unit 21 adjusts the values ​​of each operational parameter of the machine learning model 401 by repeating the first to fourth steps described above, so that for each learning dataset 310, when the training data 311 is input to the input layer 4011, the output layer 4014 outputs an output value suitable for the corresponding correct answer data 312. For example, the control unit 21 may repeat the first to fourth steps described above until, for each learning dataset 310, the sum of the errors between the output value obtained from the output layer 4014 and the correct answer data 312 becomes below a threshold. Suitable for the value of the correct answer data 312 and the output value of the output layer 4014 may also include generating an error based on such a threshold between the value of the correct answer data 312 and the output value of the output layer 4014. The threshold may be appropriately set according to the implementation method. Alternatively, the control unit 21 may repeat the first to fourth steps described above a predetermined number of times. The number of repetitions can be specified, for example, by a setting value within the learning program 821 or by operator input.

[0218] Therefore, the control unit 21 can generate a machine learning model 451 that has been trained to be suitable for the input and output values ​​of the corresponding positive solution data 312 for the input of the training data 311. For example, in the case of inferring the optimal observation method of the sensor, a machine learning model 451 that has been trained to have the ability to infer the change method from the current observation state of the sensor to an appropriate observation state can be generated.

[0219] 1-5. Others

[0220] It should be noted that when the machine learning method employs supervised learning, the composition of the machine learning model 40 is not limited to neural networks; other models besides neural networks can also be used as the machine learning model 40. For example, the machine learning model 40 can be composed of regression models, support vector machines, decision trees, etc. The supervised learning method is also not limited to the examples mentioned above; it can be appropriately selected based on the composition of the machine learning model 40.

[0221] 2. Second case

[0222] Figure 7B The diagram schematically illustrates the structure of machine learning model 40 and a second example of a machine learning method. In this second example, machine learning model 40 employs a generative model, and the machine learning method uses unsupervised learning. It should be noted that... Figure 7B For ease of explanation, an example of each of the local learning data 30, the machine learning model 40, and the machine learning model 45 after learning will be described as local learning data 302, machine learning model 402, and machine learning model 452 after learning.

[0223] 2-1. Example of a Machine Learning Model

[0224] In the second example, machine learning model 402 is configured to generate pseudo-data 325 based on noise 323. Machine learning model 402 is used as a generative model. The machine learning of machine learning model 402 is performed together with other machine learning models 412. The other machine learning models 412 are configured to identify the source of the provided input data, that is, to identify whether the provided input data is the training data 321 included in the local learning data 302 or the pseudo-data 325 generated by machine learning model 402. The other machine learning models 412 are used as identification models.

[0225] In the second example, adversarial learning is implemented between machine learning model 402 and other machine learning models 412. Machine learning models 402 and 412 each possess the same computational parameters as machine learning model 401. The configuration of each of the machine learning models 402 and 412 is not particularly limited, as long as it can perform each information processing step, and can be appropriately determined according to the implementation method. Machine learning models 402 and 412 can each be constructed from neural networks. Furthermore, machine learning models 402 and 412 can each be configured to further receive input data representing conditions (which can be called labels). Thus, conditions can be defined for data generation and recognition.

[0226] 2-2. Example of Local Learning Data Composition

[0227] In adversarial learning, the local learning data 302 consists of multiple training data sets 321. Through adversarial learning, the machine learning model 402 acquires the ability to generate pseudo-data 325 that is suitable (e.g., similar to) the training data 321 included in the local learning data 302. Therefore, the training data 321 consists of data that the machine learning model 402 wants to generate through the capabilities acquired via adversarial learning.

[0228] For example, in the case where the machine learning model 402 is able to predict the movement path of a mobile entity (i.e., generate a predicted movement path for the mobile entity), the training data 321 may consist of data representing the actual or virtual movement path of the mobile entity. Alternatively, for example, in the case where the machine learning model 402 is able to infer a conversational strategy suitable for a user, the training data 321 may consist of data representing the conversational strategies actually or virtually adopted between any users. When the machine learning model 402 and other machine learning models 412 are respectively configured to further receive input data representing conditions, the training data 321 may also be associated with the data representing conditions.

[0229] 2-3. Regarding step S101

[0230] In step S101 described above, the control unit 21 acquires local learning data 302, which consists of multiple training data sets 321. The method for generating the training data 321 is not particularly limited and can be appropriately selected depending on the implementation method. For example, similar to the first example described above, by actually or virtually creating various situations for which data is to be generated, data exhibiting the created various situations can be acquired as training data 321. The acquired training data 321 can also be associated with data representing conditions (e.g., categories) used to identify the situation.

[0231] Training data 321 can be automatically generated by computer operations or manually generated by an operator. Furthermore, the generation of training data 321 can be performed by each local learning device 2 or by a computer other than the local learning devices 2. When training data 321 is generated by each local learning device 2, the control unit 21 automatically or manually executes the aforementioned series of generation processes via the operator's operation through the input device 25, thereby acquiring local learning data 302 composed of multiple training data 321s. On the other hand, when training data 321 is generated by other computers, the control unit 21 acquires local learning data 302 composed of multiple training data 321s generated by other computers, for example, via a network, storage medium 92, etc. It is also possible for each local learning device 2 to generate a portion of training data 321, and for one or more other computers to generate other training data 321s. The number of training data 321s acquired is not particularly limited and can be appropriately selected according to the implementation method.

[0232] 2-4. Regarding step S102

[0233] In step S102 above, the control unit 21 performs adversarial learning on the machine learning model 402 and the other machine learning model 412 using the acquired local learning data 302. An example of the adversarial learning process will be described below. For ease of explanation, it is assumed that the machine learning model 402 and the other machine learning model 412 are each composed of a neural network in the following process. However, the composition of the machine learning model 402 and the other machine learning model 412 is not limited to this example. At least one of the machine learning model 402 and the other machine learning model 412 may be composed of a model other than a neural network.

[0234] Preprocessing

[0235] As preprocessing, the control unit 21 prepares machine learning model 402 and other machine learning models 412 as the objects of processing. The structure of each of the prepared machine learning model 402 and other machine learning models 412 and the initial values ​​of each operation parameter can be provided by templates or by operator input. In addition, in the case of relearning, the control unit 21 can also prepare machine learning model 402 and other machine learning models 412 based on the learning result data obtained through past supervised learning.

[0236] Machine learning for recognition models

[0237] First, the control unit 21 executes machine learning from other machine learning models 412 based on the fixed computational parameters of the machine learning model 402. That is, the control unit 21 extracts multiple noises 323 from a predetermined probability distribution (e.g., a Gaussian distribution). Next, the control unit 21 provides each extracted noise 323 to the machine learning model 402 and performs computational processing on the machine learning model 402. In other words, the control unit 21 inputs each noise into the input layer of the machine learning model 402 and sequentially determines the firing of each neuron included in each layer, starting from the input side. Thus, the control unit 21 obtains pseudo-data 325 generated based on the noise 323 as the output from the output layer of the machine learning model 402. The number of generated pseudo-data 325 is not particularly limited and can be appropriately selected according to the implementation method.

[0238] Then, the control unit 21 uses the multiple training data sets 321 and the generated pseudo data sets 325 included in the local learning data 302 to perform learning processing on other machine learning models 412. In this learning processing, the other machine learning models 412 are trained to acquire the ability to identify the source of the provided input data; in other words, to identify whether the provided input data is training data 321 or pseudo data 325. This learning processing can, for example, use stochastic gradient descent, mini-batch gradient descent, etc.

[0239] Specifically, the control unit 21 inputs the pseudo-data 325 generated by the machine learning model 402 into other machine learning models 412 and performs the computational processing of the other machine learning models 412. That is, the control unit 21 inputs the pseudo-data 325 into the input layer of the other machine learning models 412 and sequentially performs firing determination of each neuron included in each layer, starting from the input side. As a result, the control unit 21 obtains the output value corresponding to the result of identifying the source of the pseudo-data 325 provided by the other machine learning models 412, and uses it as the output of the output layer of the other machine learning models 412.

[0240] In this scenario, since the input data is pseudo-data 325, other machine learning models 412 identify it as "pseudo" but a correct solution. For each piece of pseudo-data 325 generated by the machine learning model 402, the control unit 21 calculates the error between the output value obtained from the output layer and the correct solution. Similar to the first example above, the error can be calculated using a loss function.

[0241] Furthermore, the control unit 21 inputs the training data 321 included in the local learning data 302 into other machine learning models 412 and performs the computational processing of the other machine learning models 412. That is, the control unit 21 inputs the training data 321 into the input layer of the other machine learning models 412 and sequentially performs firing determination of each neuron included in each layer, starting from the input side. As a result, the control unit 21 obtains the output value corresponding to the result of recognizing the source of the provided training data 321 from the other machine learning models 412, and uses it as the output of the output layer of the other machine learning models 412.

[0242] In this scenario, since the input data is training data 321, other machine learning models 412 identify it as "true," which is the correct solution. The control unit 21 calculates the error between the output value obtained from the output layer and the correct solution for each piece of training data 321 included in the local learning data 302. Similarly, the error can be calculated using a loss function.

[0243] Similar to the first example described above, control unit 21 uses the gradient of the calculated error in the output value to calculate the error in the values ​​of the operational parameters of other machine learning models 412 via backpropagation. Based on the calculated error, control unit 21 updates the values ​​of the operational parameters of the other machine learning models 412. The degree of updating the operational parameter values ​​can be adjusted by the learning rate.

[0244] The control unit 21 extracts samples from multiple training data sets 321 included in the local learning data 302 and multiple pseudo data sets 325 generated by the machine learning model 402, and performs the aforementioned series of learning processes using the extracted samples. As a result, the control unit 21 adjusts the values ​​of the operational parameters of other machine learning models 412. The number of times this adjustment is repeated can be appropriately set.

[0245] For example, similar to the first example described above, control unit 21 can repeatedly perform a series of learning processes until the sum of errors in the calculated output values ​​falls below a threshold. Alternatively, control unit 21 can repeat the adjustment of the values ​​of the calculation parameters based on the aforementioned series of learning processes a predetermined number of times. In this case, the number of repeated adjustments can be specified, for example, by a setting within the learning program 821 or by operator input.

[0246] Therefore, the control unit 21 is able to construct other machine learning models 412 that have the ability to identify the source of the provided input data based on the multiple training data 321 included in the local learning data 302 and the multiple pseudo data 325 generated by the machine learning model 402.

[0247] Machine learning for generative models

[0248] Next, the control unit 21 executes machine learning of the machine learning model 402 while fixing the values ​​of the computational parameters of the other machine learning models 412. That is, the control unit 21 uses the other machine learning models 412 that have been trained to perform the learning process of the machine learning model 402. In this learning process, the control unit 21 trains the machine learning model 402 to generate data (pseudo-data 325) that reduces the recognition performance of the other machine learning models 412. In other words, the control unit 21 trains the machine learning model 402 to generate data that causes the other machine learning models 412 to misidentify as "true" (i.e., data from local learning data 302).

[0249] Specifically, firstly, similarly as described above, the control unit 21 extracts multiple noises 323 from a predetermined probability distribution. Next, the control unit 21 provides each extracted noise 323 to the machine learning model 402 and performs computational processing on the machine learning model 402. In other words, the control unit 21 inputs each noise into the input layer of the machine learning model 402 and sequentially determines the firing of each neuron included in each layer, starting from the input side. Thus, the control unit 21 obtains pseudo-data 325 generated based on the noise 323 as the output from the output layer of the machine learning model 402. The number of generated pseudo-data 325 is not particularly limited and can be appropriately selected according to the implementation method. It should be noted that in the learning processing of the machine learning model 402, the pseudo-data 325 generated during the machine learning of other machine learning models 412 can also be used. In this case, the series of processes for generating pseudo-data 325 can be omitted in the learning processing of the machine learning model 402.

[0250] Next, the control unit 21 provides the generated pseudo-data 325 to other machine learning models 412 and performs the computational processing of the other machine learning models 412. In other words, the control unit 21 inputs the pseudo-data 325 into the input layer of the other machine learning models 412 and sequentially performs firing determination of each neuron included in each layer, starting from the input side. As a result, the control unit 21 obtains the output value corresponding to the result of the other machine learning models 412 identifying the source of the provided pseudo-data 325, and uses it as the output of the output layer of the other machine learning models 412.

[0251] In this scenario, other machine learning models 412 misidentify the input data as "true," which is the correct answer. For each piece of pseudo-data 325 generated by machine learning model 402, control unit 21 calculates the error between the output value obtained from the output layer of the other machine learning model 412 and the correct answer. Thus, control unit 21 calculates the error in the output value to reduce the recognition performance of the other machine learning model 412. Similarly, the error can be calculated using a loss function.

[0252] The control unit 21 uses the error backpropagation method to backpropagate the gradient of the calculated output value error to the operational parameters of the machine learning model 402 via other machine learning models 412, and calculates the error in the values ​​of the operational parameters of the machine learning model 402. Based on the calculated error, the control unit 21 updates the values ​​of the operational parameters of the machine learning model 402. Similarly, the degree of updating the values ​​of the operational parameters can be adjusted by the learning rate.

[0253] The control unit 21 adjusts the values ​​of the computational parameters of the machine learning model 402 by repeatedly executing the aforementioned series of learning processes. The number of times this adjustment is repeated can be appropriately set. Similarly, the control unit 21 can repeat the aforementioned series of learning processes until the sum of errors in the calculated output values ​​falls below a threshold, or the adjustment of the computational parameters based on the aforementioned series of learning processes can be repeated a predetermined number of times. Thus, the control unit 21 can train the machine learning model 402 to generate data that reduces the recognition performance of the other trained machine learning models 412.

[0254] • Repeated execution of each learning process

[0255] The control unit 21 alternately and repeatedly executes the learning processes of the other machine learning model 412 and the machine learning model 402. The number of times the processes are alternately and repeatedly executed can be appropriately set. The number of times the processes are alternately and repeatedly executed can be specified, for example, by a setting value within the learning program 821, or by input from the operator. It should be noted that when the data representing the conditions is associated with the training data 321, in each of the above learning processes, the data representing the conditions can be further input into the machine learning model 402 and the other machine learning models 412, respectively.

[0256] Therefore, the accuracy of other machine learning models 412 and 402 can be improved alternately. As a result, machine learning model 402 gains the ability to generate pseudo-data 325 suitable for the training data 321 included in the local learning data 302. In other words, control unit 21 can generate a fully learned machine learning model 452 that has gained the ability to generate pseudo-data 325 suitable for the training data 321 included in the local learning data 302. For example, in the case of predicting the movement path of a moving object, a fully learned machine learning model 452 that has gained the ability to generate data representing the predicted movement path of the moving object can be generated.

[0257] 2-5. Others

[0258] It should be noted that when unsupervised learning is used in the machine learning method, the composition of the machine learning model 40 is not limited to the generative model mentioned above; other models besides generative models can also be used as the machine learning model 40. The machine learning method can include clustering, etc. Furthermore, the machine learning model 40 can be composed of single-class support vector machines, self-organizing maps, other recognition models, etc.

[0259] 3. The third case

[0260] Figure 7C The diagram schematically illustrates the structure of machine learning model 40 and a third example of a machine learning method. In this third example, reinforcement learning is employed as the machine learning method. It should be noted that... Figure 7C For ease of explanation, an example of each of the local learning data 30, the machine learning model 40, and the machine learning model 45 after learning will be described as local learning data 303, machine learning model 403, and machine learning model 453 after learning.

[0261] 3-1. Example of a Machine Learning Model

[0262] In the third example, the machine learning model 403 can be value-based, policy-based, or a combination of both. When using a value-based approach, the machine learning model 403 can be composed of value functions such as a state value function and an action value function (Q-function). The state value function outputs the value of the given state. The action value function outputs the value of each action given the given state. When using a policy-based approach, the machine learning model 403 can be composed of policy functions, which output the probability of choosing each action given the given state. When using both approaches, the machine learning model 403 can be composed of a value function (Critic) and a policy function (Actor). These functions can be represented, for example, by data tables, functional expressions, etc. When represented by functional expressions, the functions can be constructed using neural networks, linear functions, decision trees, etc.

[0263] 3-2. Example of Local Learning Data Composition

[0264] In reinforcement learning, it is essentially assumed that an agent interacts with the learning environment by acting according to a policy. The agent's entity is, for example, a CPU. A machine learning model 403 performs actions according to the aforementioned configuration, using a policy to determine the actions. The agent observes the states related to the reinforcement actions within the provided learning environment.

[0265] The state of the observed object and the action performed can be appropriately set according to the inferences learned by the machine learning model 403. For example, in the case of inferring the optimal observation method of the sensor, the state of the observed object can be the observation state of the sensor, and the action performed can be changing the observation state of the sensor. Alternatively, in the case of predicting the movement path of a moving body, the state of the observed object can be a state related to the movement path of the moving body, and the action performed can be moving the moving body. Furthermore, in the case of inferring a conversational strategy suitable for the user, the state of the observed object can be a state related to the user's conversational actions, and the action performed can be engaging in conversation with the user. Additionally, in the case of inferring motion instructions for a robot device suitable for a task, the state of the observed object can be a state related to the state of at least one of the robot device's internal and external conditions, and the action performed can be an action based on the motion instructions. The learning environment can be appropriately created manually or automatically.

[0266] The agent uses machine learning model 403 to perform a prescribed inference on the current state of the observation (input data) and determines the action to be taken based on the result of the inference. Alternatively, the action can be determined randomly. When the determined action is executed, the state of the observation transitions to the next state. Depending on the circumstances, the agent can receive immediate rewards based on the learned environment.

[0267] While repeatedly determining and executing the action, the agent updates the machine learning model 403 to maximize the sum of immediate rewards (i.e., value). Thus, the optimal action, the one expected to yield high value, is reinforced, resulting in a strategy that makes such an action possible (the learned machine learning model 453).

[0268] Therefore, in reinforcement learning, the local learning data 303 consists of state transition data, which is obtained through trial and error, representing the state transition from the current state to the next state through the action performed, and receiving an immediate reward as appropriate. A set of state transition data can consist of data representing the trajectory of all state transitions in one round, or it can consist of data representing a specified number of state transitions (more than once).

[0269] Furthermore, immediate rewards can be calculated based on state transitions using reward functions. Reward functions can be represented by data tables, functional expressions, or rules. When represented by functional expressions, reward functions can be constructed from neural networks, linear functions, decision trees, etc. Reward functions can also be manually set by the operator or others based on the inferences learned by the machine learning model 403.

[0270] Alternatively, the reward function can be set to provide immediate rewards based on the appropriateness of the inference result (i.e., the determined action) of the machine learning model 403. The appropriateness of the inference result can also be represented by a predetermined number of levels, such as five or ten. Alternatively, the appropriateness of the inference result can also be represented by continuous values. Furthermore, the appropriateness of the inference result can be manually provided by an operator, etc. Alternatively, the appropriateness of the inference result can be evaluated using a judge 48 according to a predetermined benchmark. The predetermined benchmark can be appropriately set according to the content of the inference. The judge 48 can be appropriately configured to evaluate the appropriateness of the inference result according to the predetermined benchmark.

[0271] As a specific example, in the case of performing image inspection of products, imagine a scenario where reinforcement learning enables a machine learning model 403 to learn the ability to infer the optimal observation method for a camera to detect defects in the product. The scenario of inferring the optimal observation method for the camera is an example of the scenario of inferring the optimal observation method for a sensor. As a reinforcement learning environment, multiple products, each containing different defects, can be prepared, and an environment can be created where the prepared products are photographed by a camera.

[0272] In this case, the defined benchmark could be a benchmark regarding whether product defects can be detected based on image data obtained from a camera. Correspondingly, the decision maker 48 could be configured as a detector capable of detecting product defects based on image data. The detector could be configured to detect defects through image processing such as edge detection, or it could be configured as a machine learning model that has been trained, such as a neural network.

[0273] In this case, the reward function can be set to calculate an immediate reward based on the detector's accuracy in detecting defects. For example, the detector can be tested multiple times on image data obtained from the camera. The reward function can be set to provide a positive immediate reward based on the results of these multiple defect detection trials; if the success rate of defect detection is above a threshold, a negative immediate reward (penalty) is provided. It should be noted that the number of trials and the threshold used as the basis for providing a positive immediate reward can be provided separately through program settings or manually by the operator.

[0274] Similarly, in other situations, rewards can be set based on the appropriateness of the inference results. For example, in predicting the movement path of a mobile object, the prediction of the movement path can be tested multiple times. The reward function can be set based on the results of these multiple predictions, providing a positive immediate reward if the prediction accuracy is above a threshold, and a negative immediate reward (penalty) if the prediction accuracy is below an acceptable value. Similarly, in inferring a conversational strategy suitable for the user, the inference of a suitable conversational strategy can be tested multiple times. The reward function can also be set based on the results of these multiple inferences, providing a positive immediate reward if the inference accuracy is above a threshold, and a negative immediate reward (penalty) if the inference accuracy is below an acceptable value. The same reward function can be set in the case of inferring action commands for a robot device suitable for the task. It should be noted that the method for evaluating appropriateness is not limited to the method based on the accuracy of the inference results described above. For example, in the scenario of inferring the optimal observation method for a sensor, it is envisioned that the shorter the time spent by the sensor to measure the same object, the better. Correspondingly, the reward function can also be set such that the longer the sensor takes to measure, the more negative the immediate reward, and the shorter the sensor takes to measure, the more positive the immediate reward. Thus, the reward function can be set such that the more likely the desired result is to be obtained, the higher the immediate reward, and vice versa.

[0275] Alternatively, the reward function can be inferred from case data obtained from experts via inverse reinforcement learning. Case data consists of data representing the expert's demonstration (trajectory). In scenarios involving inferring the optimal observation method for sensors, case data may, for example, consist of data representing the sensor's modification method obtained through the skilled operator's actions. In scenarios involving predicting the movement path of a mobile object, case data may, for example, consist of data representing the actual path the mobile object moves along. In scenarios involving inferring a conversational strategy suitable for a user, case data may, for example, consist of data representing the conversational strategy specified by the skilled operator. In scenarios involving inferring action commands for a robot device suitable for a task, it may consist of data representing the skilled operator's operational trajectory on the robot device. The method for generating case data is not particularly limited and can be appropriately selected depending on the implementation method. For example, case data can be generated by recording the expert's demonstration trajectory using sensors, etc.

[0276] The method of inverse reinforcement learning is not particularly limited and can be appropriately selected according to the implementation method. Inverse reinforcement learning can, for example, use methods based on the maximum entropy principle, methods based on the minimization of relative entropy, or methods utilizing generative adversarial networks (e.g., Justin Fu, et al., "Learning Robust Rewards with Adversarial Inverse Reinforcement Learning", arXiv: 1710.11248, 2018). When the reward function is obtained through inverse reinforcement learning, the local learning data 303 can further incorporate the instance data used in inverse reinforcement learning.

[0277] 3-3. Regarding steps S101 and S102

[0278] In step S101 above, the control unit 21 may also use the machine learning model 403 that has completed training to obtain the state transition data by executing the trial error. Alternatively, the state transition data may be generated by another computer. In this case, in step S101 above, the control unit 21 may also obtain the state transition data generated by another computer via a network, storage medium 92, etc.

[0279] In step S102 above, the control unit 21 updates the values ​​of the operational parameters of the machine learning model 403 based on the obtained state transition data to maximize its value. The method for adjusting the values ​​of the operational parameters of the machine learning model 403 can be appropriately selected according to the configuration of the machine learning model 403. For example, if the machine learning model 403 is composed of a neural network, the values ​​of the operational parameters of the machine learning model 403 can be adjusted using methods such as backpropagation of errors, in the same way as in the first and second examples described above.

[0280] Control unit 21 repeats steps S101 and S102, adjusting the values ​​of the operational parameters of machine learning model 403 to maximize the obtained value (expected value) (e.g., until the update amount falls below a threshold). That is, training machine learning model 403 involves repeatedly correcting the values ​​of the operational parameters constituting machine learning model 403 to obtain more reward until a predetermined condition is met. Thus, control unit 21 can generate a fully trained machine learning model 453 that has acquired the ability to execute predetermined inferences (in the third example, determining actions that can be expected to yield high value). For example, in the case of inferring the optimal observation method for the aforementioned sensor, a fully trained machine learning model 453 can be generated that has acquired the ability to infer a change method from the current observation state of the sensor to an appropriate observation state.

[0281] When the machine learning model 403 is based on value, the reinforcement learning methods mentioned above can include TD (temporal difference), TD(λ), Monte Carlo methods, dynamic programming, etc. The action determined in the trial error can be either the same policy or a different policy. As specific examples, reinforcement learning methods such as Q-learning and Sarsa can be used. In the trial error, a random action with probability ε can also be adopted (ε-greedy method).

[0282] Furthermore, when the machine learning model 403 is policy-based, the aforementioned reinforcement learning method can utilize the policy gradient method, etc. In this case, the control unit 21 calculates the gradient of the operational parameters of the policy function in the direction of the obtained value increase, and updates the values ​​of the operational parameters of the policy function based on the calculated gradient. The calculation of the gradient of the policy function can, for example, use the REINFORCE algorithm, etc.

[0283] Furthermore, when the machine learning model 403 consists of both, the aforementioned reinforcement learning method can use the Actor Critic method, etc.

[0284] Furthermore, in the case of implementing inverse reinforcement learning, before performing the reinforcement learning process described above, the control unit 21 further acquires case data in step S101. The case data can be generated by each local learning device 2 or by other computers. If generated by other computers, the control unit 21 can also acquire the case data generated by other computers via a network, storage medium 92, etc. Next, the control unit 21 uses the acquired case data to set a reward function by performing inverse reinforcement learning. Then, the control unit 21 uses the reward function set through inverse reinforcement learning to perform the reinforcement learning process described above. Thus, the control unit 21, using the reward function set through inverse reinforcement learning, is able to generate a fully learned machine learning model 453 that has acquired the ability to perform predetermined inferences.

[0285] 4. Summary

[0286] In this embodiment, the machine learning model 40 can be configured using at least any of the three configurations described above. The control unit 21 can generate a fully trained machine learning model 45 that has acquired the ability to perform a predetermined inference (trained to perform a predetermined inference) by employing at least any of the three machine learning methods described above. When the fully trained machine learning model 45 is generated, the control unit 21 causes the process to proceed to the next step, S103.

[0287] (Step S103)

[0288] In step S103, the control unit 21 operates as the environmental data acquisition unit 212, acquiring learning environment data 35 related to the environment in which the local learning data 30 is obtained. Attributes related to the environment in which the local learning data 30 is obtained can include all phenomena that may be associated with the inference, such as attributes of objects or users that are directly or indirectly related to the predetermined inference. The learning environment data 35 is the same type of data as the environmental data 63 obtained by the inference device 1.

[0289] The content of the learning environment data 35 is not particularly limited and can be appropriately selected based on the content of the environment considered in the integration processing of the inference device 1. For example, in the case of inferring the optimal observation method for the sensor, the learning environment data 35 may include information related to environmental attributes (e.g., brightness, temperature, etc.) that affect the observation of the sensor. Additionally, for example, in the case of inferring a conversational strategy suitable for the user, the learning environment data 35 may include information related to the user's attributes (e.g., age, gender, occupation, place of birth, personality type, etc.).

[0290] Furthermore, the method for acquiring the learning environment data 35 is not particularly limited and can be appropriately selected depending on the implementation method. For example, the learning environment data 35 can also be acquired via input device 25 through input from an operator such as a user. Alternatively, sensors can be used in acquiring the learning environment data 35, for example.

[0291] When a sensor is used to acquire the learning environment data 35, the learning environment data 35 can be the observation data obtained by the sensor itself, or it can be data obtained by performing some information processing (e.g., feature extraction) on the observation data. As an example of performing information processing on the observation data, in the case of inferring a conversational strategy suitable for the user, consider acquiring information related to the user's gender as the learning environment data 35. In this case, the control unit 21 can also use a camera as a sensor to acquire the learning environment data 35, and acquire image data of the user's face. Then, the control unit 21 can also perform image processing to infer gender based on the face on the acquired image data. Based on the result of this inference processing, the control unit 21 can also acquire information related to the user's gender.

[0292] Furthermore, the path for obtaining the learning environment data 35 is not particularly limited and can be appropriately selected according to the implementation method. The control unit 21 may also directly obtain the learning environment data 35 via the external interface 24, input device 25, etc. Alternatively, the control unit 21 may indirectly obtain the learning environment data 35 via a network, storage medium 92, etc.

[0293] When the learning environment data 35 is acquired, the control unit 21 initiates the next step, S104. It should be noted that the timing of executing step S103 is not limited to this example. Step S103 can be executed at any time before proceeding to the next step, S104.

[0294] (Step S104)

[0295] The control unit 21, acting as the storage processing unit 214, generates information related to the learned machine learning model 45 as learning result data 47. Figures 7A to 7C In this example, the control unit 21 generates information related to the learned machine learning models 451-453 as learning result data 47. The information related to the learned machine learning models 45 may include, for example, information representing the structure of the learned machine learning models 45, and information representing the values ​​of the operational parameters adjusted through machine learning. Then, the control unit 21 stores the generated learning result data 47 in a designated storage area in association with the learning environment data 35.

[0296] The designated storage area may be, for example, RAM within the control unit 21, storage unit 22, external storage device, storage media, or a combination thereof. Storage media may be, for example, CDs, DVDs, etc., and the control unit 21 may also store the learning result data 47 in the storage media via drive 27. External storage devices may be, for example, data servers such as NAS (Network Attached Storage). In this case, the control unit 21 may also use communication interface 23 to store the learning result data 47 in the data server via a network. Alternatively, the external storage device may be, for example, an external storage device connected to each local learning device 2.

[0297] Therefore, when the learning result data 47 is saved, the control unit 21 ends a series of information processing related to the generation of the machine learning model 45 after learning.

[0298] It should be noted that the learning result data 47 and learning environment data 35 can be provided to the inference device 1 at any time. For example, the control unit 21 can forward the learning result data 47 and learning environment data 35 to the inference device 1 as part of step S104 or separately from the processing of step S104. The control unit 11 of the inference device 1 can also obtain the learning result data 47 and learning environment data 35 by receiving this forwarding. Alternatively, for example, the control unit 11 can also access each local learning device 2 or data server via the network using the communication interface 13 to obtain the learning result data 47 and learning environment data 35. Alternatively, for example, the control unit 11 can also obtain the learning result data 47 and learning environment data 35 via the storage medium 91. Alternatively, for example, the learning result data 47 and learning environment data 35 can also be pre-embedded in the inference device 1.

[0299] Furthermore, the control unit 21 can periodically update or generate new learning result data 47 and learning environment data 35 by periodically repeating the above steps S101 to S104. During this repetition, changes, corrections, additions, deletions, etc., of data included in the local learning data 30 can be performed appropriately. Then, the control unit 21 can also periodically update the learning result data 47 and learning environment data 35 held by the inference device 1 by providing updated or newly generated learning result data 47 and learning environment data 35 to the inference device 1 each time learning processing is performed.

[0300] (B) Inference Processing

[0301] Next, use Figure 8 Examples of actions related to the inferences specified in each local learning device 2 will be explained. Figure 8 This is a flowchart illustrating an example of the processing procedure related to the prescribed inference of each local learning device 2. However, the processing procedures described below are merely examples, and the steps can be modified as much as possible. Furthermore, for the processing procedures described below, steps can be omitted, substituted, or added appropriately according to the implementation method.

[0302] (Step S111)

[0303] In step S111, the control unit 21 operates as the object data acquisition unit 216 to acquire object data 225, which is the object of the predetermined inference. The object data 225 is the same type of data as the object data 61 obtained by the inference device 1.

[0304] The content of object data 225 is not particularly limited and can be appropriately selected based on the content of the prescribed inference. For example, in the case of inferring the optimal observation method for the sensor, object data 225 may include information representing attributes related to the sensor's setup (e.g., the sensor's setup angle, the distance between the sensor and the observed object, etc.) as information related to the sensor's current observation state. Additionally, for example, in the case of inferring a conversational strategy suitable for the user, object data 225 may include data related to the user's conversational actions obtained through input devices (e.g., keyboard, touch panel), camera, microphone, etc. If the learned machine learning model 45 is composed of a generative model, object data 225 may include noise extracted from a prescribed probability distribution.

[0305] Furthermore, the method for acquiring object data 225 is not particularly limited and can be appropriately selected depending on the implementation method. For example, object data 225 can also be acquired via input device 25 through input from an operator such as a user. Alternatively, sensors can be used, for example, in acquiring object data 225.

[0306] When a sensor is used to acquire object data 225, object data 225, like learning environment data 35, can be the observation data itself obtained by the sensor, or data obtained by performing some information processing on the observation data. As an example of performing information processing on the observation data, consider a scenario where a conversational strategy suitable for the user is inferred, where the string of the user's speech content is acquired as object data 225. In this case, control unit 21 can also use a microphone as a sensor to acquire object data 225, acquiring sound data including the user's speech. Then, control unit 21 can also perform speech analysis on the acquired sound data. Based on the result of this speech analysis processing, control unit 21 can also acquire the string of the user's speech content.

[0307] Furthermore, when sensors are used in acquiring both the learning environment data 35 and the object data 225, the learning environment data 35 and the object data 225 can originate from the same observation data obtained from the same sensor, or from different observation data obtained from different sensors. As an example of a scenario where the data originates from the same observation data, in the case of inferring a conversational strategy suitable for the user, a scenario using a camera is envisioned. In this case, the control unit 21 can also infer the user's gender based on image data obtained from the camera, and acquire information related to the inferred user's gender as the learning environment data 35. Additionally, the control unit 21 can also acquire the string of the user's speech content as the object data 225 by analyzing the string of the user's speech content based on sound data obtained from the camera.

[0308] Furthermore, the path for obtaining the object data 225 is not particularly limited and can be appropriately selected according to the implementation method. The control unit 21 may also directly obtain the object data 225 via the external interface 24, input device 25, etc. Alternatively, the control unit 21 may indirectly obtain the object data 225 via a network, storage medium 92, etc.

[0309] When object data 225 is acquired, the control unit 21 causes the processing to proceed to the next step S112.

[0310] (Step S112)

[0311] In step S112, the control unit 21 operates as the inference unit 217, performing a predetermined inference on the acquired object data 225 using the inference model derived from the local learning data 30. In this embodiment, the control unit 21 maintains the learning result data 47, thereby possessing a learned machine learning model 45 as an inference model. The control unit 21 sets the learned machine learning model 45 with reference to the learning result data 47. Next, the control unit 21 provides the object data 225 to the learned machine learning model 45 and performs computational processing on the learned machine learning model 45.

[0312] The computational processing can be performed appropriately based on the structure of the learned machine learning model 45. If the learned machine learning model 45 is composed of a function, the control unit 21 substitutes the object data 225 into the function and performs the computational processing of that function. If the learned machine learning model 45 is composed of a neural network, the control unit 21 inputs the object data 225 into the input layer and sequentially determines the firing of each neuron included in each layer, starting from the input side. If the learned machine learning model 45 is composed of a data table, the control unit 21 compares the object data 225 with the data table.

[0313] Therefore, the control unit 21 obtains the result of performing a prescribed inference on the object data 225 as the output of the learned machine learning model 45 (i.e., the execution result of the computational processing). The obtained inference result depends on the ability mastered by the learned machine learning model 45. For example, if the learned machine learning model 45 has the ability to infer the optimal observation method for the sensor, the control unit 21 can obtain information related to the change method for changing the observation state of the sensor to an appropriate observation state based on the current observation state of the sensor as the result of the inference processing. When the inference result is obtained, the control unit 21 causes the processing to proceed to the next step S113.

[0314] (Step S113)

[0315] In step S113, the control unit 21 operates as an output unit 218, outputting information related to the inference result of the object data 225.

[0316] The output destination and the content of the output information can be appropriately determined according to the implementation method. For example, the control unit 21 may directly output the result of performing a predetermined inference on the object data 225 through step S112 to the output device 26. Alternatively, the control unit 21 may perform certain information processing based on the result of performing the predetermined inference. Then, the control unit 21 may output the result of performing this information processing as information related to the inference result. The output of the result of performing this information processing may include outputting a specific message based on the inference result, controlling the operation of the object device based on the inference result, etc. The output destination may be, for example, the output device 26, the output device of another computer, the object device being controlled, etc.

[0317] When the output of information related to the inference result is completed, the control unit 21 ends the series of information processing related to the prescribed inference. It should be noted that, within the prescribed period, the control unit 21 may also continue to repeat the series of information processing steps S111 to S113. The timing of repetition can be arbitrary. Therefore, each local learning device 2 can continue to implement the prescribed inference.

[0318] [Inference apparatus]

[0319] Next, use Figure 9A Examples of actions related to the inference specified by inference device 1 will be explained. Figure 9A This is a flowchart illustrating an example of the processing steps related to the prescribed inference of the inference device 1. The processing steps described below are an example of the "inference method" of the present invention. However, each processing step described below is merely an example, and the steps can be modified as much as possible. Furthermore, regarding each processing step described below, steps can be appropriately omitted, substituted, or added according to the implementation method.

[0320] (Steps S201 and S202)

[0321] In step S201, the control unit 11 operates as the first data acquisition unit 111 to acquire the specified object data 61, which is the object of the specified inference. Except for the difference in the processing environment, the process of acquiring the object data 61 in step S201 can be the same as the process of acquiring the object data 225 in step S111 described above.

[0322] That is, the content of object data 61 can be appropriately selected based on the prescribed inference. The method and path for acquiring object data 61 can be appropriately selected according to the implementation method. Object data 61 can also be acquired through input from the operator via input device 15. The acquisition of object data 61 can also utilize the same type of sensor used in acquiring object data 225. When a sensor is used in acquiring object data 61, object data 61 can be the observation data itself obtained by the sensor, or it can be data obtained by performing some information processing on the observation data. Control unit 11 can directly acquire object data 61 via external interface 14, input device 15, etc., or it can indirectly acquire object data 61 via network, storage medium 91, etc.

[0323] In step S202, the control unit 11 operates as the second data acquisition unit 112 to acquire environmental data 63 related to the object environment in which the prescribed inference is performed. Except for the difference in the processing environment, the process of acquiring environmental data 63 in step S202 can be the same as the process of acquiring learning environment data 35 in step S103 described above.

[0324] That is, the content of environmental data 63 can be appropriately selected based on the environmental content considered in the integration processing. The method and path for acquiring environmental data 63 can be appropriately selected according to the implementation method. Environmental data 63 can also be acquired through input by the operator via input device 15. The acquisition of environmental data 63 can also utilize the same type of sensor used in acquiring learning environmental data 35. When a sensor is used in acquiring environmental data 63, environmental data 63 can be the observation data itself obtained by the sensor, or it can be data obtained by performing some information processing on the observation data. When a sensor is used in acquiring object data 61 and environmental data 63, object data 61 and environmental data 63 can originate from the same observation data obtained by the same sensor, or they can originate from different observation data obtained by different sensors. The control unit 11 can directly acquire environmental data 63 via external interface 14, input device 15, etc., or it can indirectly acquire environmental data 63 via network, storage medium 91, etc.

[0325] It should be noted that environmental data 63 is acquired regarding the object environment in which object data 61 is obtained, while learning environment data 35 is acquired regarding the environment in which local learning data 30 is obtained. Local learning data 30 contains data of the same type as object data 61 (e.g., the training data 311 mentioned above). Therefore, compared to environmental data 63, learning environment data 35 can also have expanded environmental attributes, such as time, location, objects, and users. Correspondingly, the data formats of environmental data 63 and learning environment data 35 can differ in ways that allow for comparison of their respective environments.

[0326] Therefore, the control unit 11 acquires object data 61 and environmental data 63. It should be noted that the processing order of steps S201 and S202 is arbitrary. Either step S201 or step S202 can be executed first, or they can be executed in parallel. When object data 61 and environmental data 63 are acquired, the control unit 11 initiates the next step, S203.

[0327] (Step S203)

[0328] In step S203, the control unit 11, acting as the result acquisition unit 113, provides object data 61 to multiple inference models derived from local learning data 30 obtained under different environments to perform prescribed inferences, thereby enabling each inference model to perform a prescribed inference on the object data 61. As a result, the control unit 11 acquires the inference results of each inference model on the acquired object data 61.

[0329] In this embodiment, each inference model is composed of a machine learning model 45 that has been trained and generated by each of the local learning devices 2. Furthermore, in this embodiment, the inference device 1 possesses each trained machine learning model 45 by holding each set of learning result data 47 generated by each of the local learning devices 2. Therefore, the control unit 11 sets each trained machine learning model 45 by referring to each set of learning result data 47. Next, the control unit 11 provides object data 61 to each trained machine learning model 45 and performs computational processing on each trained machine learning model 45. The computational processing on each trained machine learning model 45 can be the same as the computational processing on the trained machine learning model 45 in step S112 described above. Thus, the control unit 11 can obtain the inference result of each trained machine learning model 45 on the object data 61 as the output of each trained machine learning model 45. It should be noted that the trained machine learning model 45 used in the inference processing can be appropriately selected by the operator's specification, etc. When the inference results of each learned machine learning model 45 on the object data 61 are obtained, the control unit 11 causes the processing to proceed to the next step S204.

[0330] (Step S204)

[0331] In step S204, the control unit 11 operates as an integration unit 114, integrating the inference results of each inference model obtained in step S203 according to integration rule 5. In this embodiment, the control unit 11 integrates the inference results of each learned machine learning model 45 according to integration rule 5. As a result, the control unit 11 generates inference results for object data 61 in the object environment.

[0332] In this embodiment, the control unit 11 maintains the integration rule data 57, thereby possessing information related to the integration rule 5. The integration rule 5 has multiple integration parameters P that specify the degree of importance given to the inference results of each learned machine learning model 45 in the object environment. The control unit 11 sets the integration rule 5 by referring to the integration rule data 57. Then, the control unit 11 integrates the inference results of each learned machine learning model 45 according to the integration rule 5 through the following process.

[0333] Figure 9B This is a flowchart illustrating an example of the processing procedure of the subroutine related to the integration process of step S204. The processing of step S204 in this embodiment includes the processing of steps S211 to S213. However, the processing procedure described below is only an example, and each process can be modified as much as possible. In addition, for the processing procedure described below, steps can be omitted, replaced, or added appropriately according to the embodiment.

[0334] In step S211, the control unit 11 determines the values ​​of each integrated parameter P based on the acquired environmental data 63. Details of the method for determining the values ​​of each integrated parameter P will be described later. It should be noted that, in determining the values ​​of each integrated parameter P, in addition to considering the environmental data 63, the object data 61 can also be considered. That is, the control unit 11 can also determine the values ​​of each integrated parameter P based on both the object data 61 and the environmental data 63.

[0335] In step S212, the control unit 11 uses the determined values ​​of each integration parameter P to weight the inference results of the corresponding inference models. In step S213, the control unit 11 integrates the weighted inference results of each inference model. In this embodiment, the control unit 11 uses the determined values ​​of each integration parameter P to weight the inference results of the corresponding learned machine learning models 45. Then, the control unit 11 integrates the weighted inference results of each learned machine learning model 45.

[0336] The weighting and integration method can be any method that prioritizes inference results based on weighting, and is not particularly limited, but can be appropriately set according to the implementation method. For example, the weighted inference result of integrating the inference results of each learned machine learning model 45 can be a weighted average of the values ​​representing the inference results of each learned machine learning model 45. Alternatively, the weighted inference result of integrating the inference results of each learned machine learning model 45 can be selected by majority voting based on weighting. When the specified inference is regression, the integration of inference results can be mainly performed by the above-mentioned weighted average. Furthermore, when the specified inference is recognition, the integration of inference results can be mainly performed by the above-mentioned weighted majority voting.

[0337] When the integration in step S213 is completed, the control unit 11 terminates the processing of the subroutine related to the integration process in step S204. Thus, the control unit 11 is able to generate an inference result for the object data 61 within the object environment. When the inference result is generated through this integration process, the control unit 11 causes the process to proceed to the next step, S205.

[0338] (Step S205)

[0339] Return to Figure 9A In step S205, the control unit 11 operates as the output unit 115, outputting information related to the generated inference result. Except for the different processing environment, the output processing in step S205 can be the same as the output processing in step S113 described above.

[0340] That is, the output destination and the content of the output information can be appropriately determined according to the implementation method. For example, the control unit 11 may directly output the inference result generated in step S204 to the output device 16. Alternatively, the control unit 11 may perform some information processing based on the generated inference result. Then, the control unit 11 may output the result of performing this information processing as information related to the inference result. The output of the result of performing this information processing may include outputting a specific message based on the inference result, controlling the operation of the controlled object device based on the inference result, etc. The output destination may be, for example, the output device 16, the output device of another computer, the controlled object device, etc.

[0341] When the output of information related to the inference result is completed, the control unit 11 ends the series of information processing related to the prescribed inference. It should be noted that, within the prescribed period, the control unit 11 may also continue to repeat the series of information processing steps S201 to S205. The timing of the repetition can be arbitrary. Therefore, the inference device 1 can continue to perform the prescribed inference.

[0342] <Method for determining the value of the integration parameter>

[0343] Next, a specific example of the method for determining the value of each integration parameter P in step S211 above will be described. In this embodiment, the control unit 11 can determine the value of each integration parameter P by at least one of the following three methods.

[0344] (1) First method

[0345] Figure 10A An example scenario illustrating a scenario in which the values ​​of each integrated parameter P are determined using the first method is shown. In the first method, the control unit 11 utilizes a computational model for calculating the values ​​of each integrated parameter P based on environmental data 63 (and object data 61).

[0346] (1-1) Processing to determine the value of the integration parameter

[0347] In this embodiment, the control unit 11 performs machine learning on the computational model 51 using the learning data 59. The computational model 51, similar to the machine learning model 40 described above, can be represented, for example, by a data table or a function. Through machine learning, the control unit 11 constructs (generates) a fully learned computational model 52 trained to make the calculated values ​​of each integration parameter P suitable for a given inference in the object environment. In the first method, the integration rule 5 further includes the fully learned computational model 52 constructed through this machine learning. Information related to the constructed fully learned computational model 52 can be stored as at least a part of the integration rule data 57, or it can be stored separately from the integration rule data 57.

[0348] In step S211 above, the control unit 11 sets the learned computational model 52 by appropriately referring to this information. Then, the control unit 11 provides environmental data 63 to the learned computational model 52 and performs computational processing on the learned computational model 52. Considering the object data 61 further, the control unit 11 provides the object data 61 to the learned computational model 52 and performs computational processing on the learned computational model 52. The computational processing of the learned computational model 52 can be the same as the computational processing of the learned machine learning model 45 described above. Thus, the control unit 11 can obtain the values ​​of each integration parameter P for each inference model as the output of the learned computational model 52. In this embodiment, after obtaining the values ​​of each integration parameter P, the control unit 11 integrates the inference results of each learned machine learning model 45 by executing step S212 and subsequent processing.

[0349] (1-2) Machine Learning of Computational Models

[0350] Next, further use Figure 10B An example of the operation of the inference device 1 related to the generation of the learned operational model 52 will be explained. Figure 10B This is a flowchart illustrating an example of the processing steps related to the generation of the learned computational model 52 by the inference device 1. However, the processing steps described below are merely examples, and each step can be modified as much as possible. Furthermore, for each processing step described below, steps can be omitted, substituted, or added appropriately according to the implementation method.

[0351] (Steps S301 and S302)

[0352] In step S301, the control unit 11 operates as a learning data acquisition unit 116, acquiring the learning data 59 used in the machine learning of the computational model 51. In step S302, the control unit 11 operates as a learning processing unit 117, using the acquired learning data 59 to perform machine learning of the computational model 51.

[0353] The machine learning in step S302 is constructed by training the computational model 51 so that the calculated values ​​of each integration parameter P are suitable for the prescribed inference in the object environment. The machine learning of this computational model 51 can be achieved by adjusting the values ​​of the computational parameters of the computational model 51 to obtain the desired output suitable for the object environment. That is, training the computational model 51 so that the calculated values ​​of each integration parameter P are suitable for the prescribed inference in the object environment can be achieved by adjusting the values ​​of the computational parameters of the computational model 51 so that the result of integrating the inference results of each inference model (each machine learning model 45) using the calculated values ​​of each integration parameter P obtained through the processing in steps S212 and S213, based on the environmental data (and object data), is such that the values ​​of each integration parameter P are suitable for the inference results in the object environment. Through this machine learning, the control unit 11 can generate a learned computational model 52 that has acquired the ability to infer the values ​​of each integration parameter P suitable for the prescribed inference in the object environment based on the environmental data (and object data).

[0354] The configuration of the computational model 51 and the machine learning method are not particularly limited and can be appropriately determined according to the implementation method. The learned computational model 52 is used to calculate the values ​​of each integration parameter P. The calculated values ​​of each integration parameter P are used to weight the inference results of each inference model during the above integration process. In the inference device 1, the inference result is generated through the above integration process. On the other hand, the learned machine learning model 45 directly derives the inference result for the object data. Therefore, the computational model 51 (the learned computational model 52) differs from the machine learning model 40 (the learned machine learning model 45) in terms of the data processed and the process of obtaining the inference result. However, in other aspects, the configuration of the computational model 51 and the machine learning method can be the same as those of the machine learning model 40.

[0355] The computational model 51 can be represented, for example, by a data table or a function. When the computational model 51 is represented by a function, it can be constructed from, for example, a neural network, a linear function, or a decision tree. The type of neural network is not particularly limited and can be appropriately determined according to the implementation method. The neural network constituting the computational model 51 can be, for example, a fully connected neural network, a convolutional neural network, a recurrent neural network, or a combination thereof. Below, two examples each of the construction of the computational model 51 and the machine learning method will be explained.

[0356] I.First example

[0357] Figure 10CThe diagram schematically illustrates the structure of computational model 51 and a first example of a machine learning method. In this first example, similar to the first example of machine learning model 40 described above, computational model 51 employs a neural network, and the machine learning method uses supervised learning. It should be noted that... Figure 10C For ease of explanation, an example of each of the learning data 59, the operation model 51, and the completed operation model 52 is described as learning data 591, operation model 511, and completed operation model 521.

[0358] I-1. Example of a computational model

[0359] In the first example, the computational model 511 consists of a three-layer fully connected neural network. Specifically, the computational model 511, starting from the input side, sequentially includes an input layer 5111, an intermediate (hidden) layer 5112, and an output layer 5113. However, the configuration of the computational model 511, like the machine learning model 401 described above, is not limited to this example and can be appropriately determined according to the implementation method. For example, the computational model 511 may have two or more intermediate layers. Alternatively, the intermediate layer 5112 may be omitted. In addition, the computational model 511 may also include other types of layers such as convolutional layers, pooling layers, and recursive layers. In other aspects, the computational model 511 can be the same as the machine learning model 401 described above. The computational model 511 includes computational parameters such as the weights of the connections between neurons in each layer 5111 to 5113 and the thresholds of each neuron.

[0360] I-2. Examples of the composition of learning data

[0361] The learning data 591 used in the supervised learning of this computational model 511 consists of multiple learning datasets 5910, each including a combination of training environment data 5911, training object data 5912, and forward solution data 5913 (supervision signal). Training environment data 5911 is the same type of data as environment data 63 and is used as training data (input data). Training object data 5912 is the same type of data as object data 61. Considering object data 61 further, training object data 5912 can be used together with training environment data 5911 as training data (input data). Forward solution data 5913 represents the result (forward solution) of performing a prescribed inference on training object data 5912 in the object environment. Forward solution data 5913 can also be called a label.

[0362] The content of the training environment data 5911 can be appropriately selected based on the content of the environment considered in the integration process. Except for the fact that it is used for training, the training environment data 5911 can be the same as the environment data 63 described above. The content of the training object data 5912 and the forward solution data 5913 can be appropriately selected based on the content of the inferences mastered by each inference model (the learned machine learning model 45). Except for the fact that it is used for training, the training object data 5912 can be the same as the object data 61 described above. Furthermore, except for the fact that it is acquired considering the object environment in which the inference device 1 performs the prescribed inferences, the training object data 5912 and the forward solution data 5913 can be the same as the training data 311 and the forward solution data 312 described above.

[0363] I-3. Regarding step S301

[0364] In step S301 above, the control unit 11 acquires learning data 591 consisting of multiple learning datasets 5910. The method for generating each learning dataset 5910 is not particularly limited and can be appropriately selected depending on the implementation method. For example, various situations in which a predetermined inference is performed in an object environment can be created, either physically or virtually, thereby acquiring data of the same type as the environment data 63 in the created situations as training environment data 5911. Alternatively, data of the same type as the object data 61 can be acquired together with the training environment data 5911 as training object data 5912. Then, a predetermined inference is performed on the acquired training object data 5912 in the object environment. In this scenario, the predetermined inference can be performed manually by an operator or the like. Then, the result (correct solution) of performing the predetermined inference on the training object data 5912 is associated with the training environment data 5911 and the training object data 5912. Thus, each learning dataset 5910 can be generated.

[0365] I-4. Regarding step S302

[0366] In step S302 above, the control unit 11 performs machine learning (supervised learning) on ​​the computational model 511 using the acquired learning data 591. In the first example, the control unit 11 trains the computational model 511 through machine learning, such that for each learning data 591, by inputting the training environment data 5911 (and the training object data 5912) into the input layer 5111, the output layer 5113 outputs the values ​​of the integration parameters P such that the results of integrating the inference results of each inference model (the learned machine learning model 45) on the training object data 5912 using the output values ​​of the integration parameters P are suitable for the values ​​of the integration parameters P such as the correct answer data 5913. Thus, the control unit 11 generates a learned computational model 521 that has the ability to infer the values ​​of the integration parameters P suitable for the specified inference in the object environment based on the environment data (and object data).

[0367] The specific processing procedure for supervised learning can be the same as that for the machine learning model 401 described above. The control unit 11 first prepares the computational model 511, which will become the object of machine learning processing. The structure of the prepared computational model 511 and the initial values ​​of each computational parameter can be provided through a template or through operator input. Furthermore, in the case of relearning, the control unit 11 can also prepare the computational model 511 based on the learning results data obtained through past supervised learning.

[0368] Next, the control unit 11 uses the training environment data 5911 included in each learning dataset 5910 as input data and the forward pass data 5913 as supervision signals to perform learning processing on the neural network constituting the computational model 511. In this learning processing, the training object data 5912 can be further used as input data. Essentially, the learning processing of the computational model 511 can be the same as that of the machine learning model 401 described above.

[0369] In the first step, the control unit 11 inputs the training environment data 5911 to the input layer 5111 of the computational model 511 for each training dataset 5910. Considering the object data 61 further, the control unit 11 may also input the training object data 5912 to the input layer 5111 of the computational model 511. Next, the control unit 11 sequentially performs firing determination for each neuron included in each layer 5111-5113, starting from the input side. Through this computational processing, the control unit 11 obtains from the output layer 5113 the output value corresponding to the result of the computational model 511 in training calculating the values ​​of each integration parameter P based on the training environment data 5911 (and the training object data 5912).

[0370] In the second step, the control unit 11 performs the processing described in step S203 on the corresponding training data 5912. That is, the control unit 11 provides the corresponding training data 5912 to each inference model, causing each inference model to perform a prescribed inference on the training data 5912. Thus, the control unit 11 obtains the inference results of each inference model on the training data 5912. In this embodiment, the control unit 11 obtains the inference results of each learned machine learning model 45 on the training data 5912.

[0371] Next, the control unit 11 uses the values ​​of each integration parameter P obtained in the first step to perform the processing described in step S204 on the inference results of each learned machine learning model 45. That is, the control unit 11 uses the values ​​of each integration parameter P obtained in the first step to weight the inference results of each learned machine learning model 45. Then, the control unit 11 integrates the weighted inference results of each learned machine learning model 45. Thus, the control unit 11 generates the inference result in the object environment. Then, the control unit 11 calculates the error between the generated inference result and the correct answer data 5913. A loss function can be appropriately used to calculate the error.

[0372] In the third step, the control unit 11 uses the error backpropagation method to backpropagate the gradient of the calculated error from the computational model 511, thereby calculating the errors in the values ​​of various computational parameters of the computational model 511, such as the weights of connections between neurons and the thresholds of each neuron. In the fourth step, the control unit 11 updates the values ​​of each computational parameter of the computational model 511 based on the calculated errors. The degree of updating the values ​​of the computational parameters can be adjusted by the learning rate.

[0373] The control unit 11 adjusts the values ​​of each operational parameter of the computational model 511 by repeating the first to fourth steps described above, so that for each training dataset 5910, by inputting the training environment data 5911 (and the training object data 5912) into the input layer 5111, the output layer 5113 outputs a value of each integration parameter P that integrates the inference results of each inference model (the learned machine learning model 45) on the training object data 5912 using the output values ​​of each integration parameter P, such that the values ​​of each integration parameter P are suitable for the correct answer data 5913. For example, the control unit 11 may repeat the first to fourth steps described above until the sum of errors of the generated inference results for each training dataset 5910 becomes below a threshold. Alternatively, the control unit 11 may repeat the first to fourth steps described above a predetermined number of times. Thus, the control unit 11 is able to generate a learned computational model 521 that has acquired the ability to infer values ​​of each integration parameter P suitable for a predetermined inference in the object environment based on the environment data (and object data).

[0374] I-5. Other

[0375] It should be noted that when using supervised learning in machine learning, the composition of the computational model 511 is not limited to neural networks; other models besides neural networks can also be used as the computational model 511. For example, the computational model 511 can be composed of regression models, support vector machines, decision trees, etc. The supervised learning method is also not limited to the examples mentioned above; it can be appropriately selected based on the composition of the computational model 511.

[0376] II. Second Case

[0377] Figure 10D The diagram schematically illustrates the structure of computational model 51 and a second example of a machine learning method. In this second example, similar to the third example of machine learning model 40 described above, reinforcement learning is employed as the machine learning method. It should be noted that... Figure 10D For ease of explanation, an example of each of the learning data 59, the operation model 51, and the completed operation model 52 is described as learning data 592, operation model 512, and completed operation model 522.

[0378] II-1. Example of a computational model

[0379] The computational model 512 can be constructed in the same way as the machine learning model 403 described above. That is, the computational model 512 can be value-based, policy-based, or both. Each function can be represented, for example, by a data table or a functional expression. When represented by a functional expression, each function can be constructed from neural networks, linear functions, decision trees, etc.

[0380] II-2. Examples of the composition of learning data

[0381] Aside from the difference in the data processed and the process of inferring results from the environment of the derived objects, the reinforcement learning method of computational model 512 can be the same as that of machine learning model 403 described above. The learning environment can be created appropriately, either manually or automatically. In the reinforcement learning of computational model 512, the state observed by the agent corresponds to the environmental data 63 and the object data 61.

[0382] The agent uses computational model 512 to calculate the values ​​of each integration parameter P based on the observed current state (input data). Similarly to step S203 above, the agent provides the object data included in the observed current state to each inference model, enabling each inference model to perform a prescribed inference on the object data. Thus, the agent obtains the inference results of each inference model on the object data. In this embodiment, the agent obtains the inference results of each learned machine learning model 45 on the object data. Next, the agent uses the values ​​of each integration parameter P obtained from computational model 512 to weight the inference results of each learned machine learning model 45. Then, the agent integrates the weighted inference results of each learned machine learning model 45. Thus, the agent generates inference results in the object environment.

[0383] The agent determines the action to be taken based on the generated inference results. Alternatively, the action can be determined randomly. When the determined action is executed, the observed state transitions to the next state. Depending on the situation, the agent can receive immediate rewards based on the learned environment. While repeatedly determining and executing the action and learning from trial errors, the agent updates the operational model 512 to maximize the sum of immediate rewards. Thus, the optimal action is reinforced, and a policy (learned operational model 522) is obtained that calculates the values ​​of the integration parameters P that make such action choices possible.

[0384] Therefore, in the second example, learning data 592 consists of state transition data, which is obtained through the trial error and represents the state transition from the current state to the next state through the action performed, and the state transition that results in an immediate reward. Similarly to the local learning data 303 described above, a set of state transition data can consist of data representing the trajectory of all state transitions in a round, or it can consist of data representing a predetermined number of state transitions (more than once).

[0385] It should be noted that, similar to the reinforcement learning in machine learning model 403 described above, a reward function for calculating immediate rewards can be used in the reinforcement learning of computational model 512. The reward function can be represented by a data table, a functional expression, or a rule. When represented by a functional expression, the reward function can be constructed from neural networks, linear functions, decision trees, etc. The reward function can also be manually set by the operator based on the inferences derived from the aforementioned series of processes.

[0386] Alternatively, the reward function can be set to provide immediate rewards based on the appropriateness of the inference results generated through the aforementioned series of processes. The appropriateness of the inference results can also be represented by a predetermined number of levels, such as five or ten. Alternatively, the appropriateness of the inference results can be represented by continuous values. Furthermore, the appropriateness of the inference results can be manually provided by an operator, etc. Alternatively, the appropriateness of the inference results can be evaluated using a decision 49 according to a predetermined benchmark. The predetermined benchmark can be the same as that of the aforementioned machine learning model 403. Additionally, the decision 49 can be the same as the decision 48 in the reinforcement learning of the aforementioned machine learning model 403.

[0387] Alternatively, the reward function can be inferred from example data obtained by experts through inverse reinforcement learning. The content of the example data, the method of generating the example data, and the method of inverse reinforcement learning are not particularly limited and can be the same as the example data used in setting the reward function in the reinforcement learning of the aforementioned machine learning model 403. When the reward function is obtained through inverse reinforcement learning, the learning data 592 can further include the example data used in inverse reinforcement learning.

[0388] II-3. Steps S301 and S302

[0389] In step S301 above, the control unit 11 may also use the training computational model 512 to obtain the state transition data by executing the trial error. Alternatively, the state transition data may be generated by another computer. In this case, in step S301 above, the control unit 11 may also obtain the state transition data generated by another computer via a network, storage medium 91, etc.

[0390] In step S302 above, the control unit 11 updates the values ​​of the operation parameters of the operation model 512 based on the obtained state transition data to maximize the value. The method for adjusting the values ​​of the operation parameters of the operation model 512 can be appropriately selected according to the configuration of the operation model 512. For example, if the operation model 512 is composed of a neural network, the values ​​of the operation parameters of the operation model 512 can be adjusted using the error backpropagation method or the same method as in the first example above.

[0391] The control unit 11 repeats the processes of steps S301 and S302, adjusting the values ​​of the operational parameters of the operational model 512 to maximize the obtained value (expected value) (e.g., until the update amount falls below a threshold). That is, training the operational model 512 involves repeatedly correcting the values ​​of the operational parameters constituting the operational model 512 to obtain more reward until the predetermined conditions are met. Thus, the control unit 11 is able to generate a fully trained operational model 522 that has acquired the ability to infer values ​​of each integrated parameter P suitable for a predetermined inference in the object environment based on environmental data (and object data). It should be noted that the reinforcement learning method can be the same as that used in the machine learning model 403 described above.

[0392] Furthermore, in the case of implementing inverse reinforcement learning, before performing the reinforcement learning process described above, the control unit 11 further acquires case data in step S301. The case data can be generated by the inference device 1 or by another computer. If generated by another computer, the control unit 11 can also acquire the case data generated by that computer via a network, storage medium 91, or the like. Next, the control unit 11 uses the acquired case data to set a reward function by performing inverse reinforcement learning. Then, the control unit 11 uses the reward function set through inverse reinforcement learning to perform the reinforcement learning process described above. Thus, the control unit 11, using the reward function set through inverse reinforcement learning, is able to generate a learned computational model 522 that has acquired the ability to infer values ​​of each integration parameter P suitable for a given inference environment based on environmental data (and object data).

[0393] III. Summary

[0394] In this embodiment, the computational model 51 can be configured using at least one of the two configurations described above. The control unit 11 can generate a learned computational model 52 by employing at least one of the two machine learning methods described above. When the machine learning of the computational model 51 is completed and the learned computational model 52 is generated, the control unit 11 causes the process to proceed to the next step S303.

[0395] (Step S303)

[0396] Return to Figure 10B In step S303, the control unit 11 acts as the storage processing unit 118 to generate the machine learning results of the computational model 51, i.e., information related to the learned computational model 52. Figure 10C and Figure 10D In the example, the control unit 11 generates information related to the learned computational model (521, 522). The control unit 11 saves the generated information related to the learned computational model 52 in a designated storage area.

[0397] The designated storage area may be, for example, RAM within the control unit 11, storage unit 12, external storage device, storage medium, or a combination thereof. Storage medium may be, for example, a CD, DVD, etc., and the control unit 11 may also store information related to the learned computational model 52 in the storage medium via drive 17. External storage device may be, for example, a data server such as a NAS. In this case, the control unit 11 may also use communication interface 13 to store information related to the learned computational model 52 in the data server via a network. Alternatively, the external storage device may be, for example, an external storage device connected to the inference device 1.

[0398] The generated information related to the learned operational model 52 can be saved as at least a part of the integrated rule data 57, or it can be saved separately from the integrated rule data 57. Therefore, when the saving of the information related to the learned operational model 52 is completed, the control unit 11 ends a series of information processing related to the generation of the learned operational model 52.

[0399] It should be noted that the control unit 11 can also periodically update or generate the learned computational model 52 by periodically repeating the above steps S301 to S303. During this repetition, changes, corrections, additions, deletions, etc., of the data included in the learning data 59 can be performed as appropriate. Then, the control unit 11 can also use the updated or newly generated learned computational model 52 for subsequent inference processing.

[0400] In the first method, a fully trained computational model 52 is constructed using machine learning, trained to ensure that the calculated values ​​of each integration parameter P are suitable for a given inference in the object environment. Therefore, by using the values ​​of each integration parameter P calculated by the constructed fully trained computational model 52, the inference results of each trained machine learning model 45 can be appropriately integrated in a manner suitable for the object environment. Thus, according to the first method, the given inference can be executed more appropriately in the object environment.

[0401] It should be noted that the number of parameters in machine learning model 40 (the learned machine learning model 45) depends on the dimensionality of the object data (61, 225) and the form of the inference results. Therefore, the more complex either the object data (61, 225) or the inference content, the more the number of parameters in machine learning model 40 increases. In contrast, the number of parameters in computational model 51 (the learned computational model 52) depends on the dimensionality of the environment data 63 and the number of learned machine learning models 45 used for the specified inference in the object environment. Since the information processing of computational model 51 is merely determining the values ​​of each integration parameter P, even if the object data (61, 225) and the inference content become more complex, the increase in the number of parameters in computational model 51 can be suppressed.

[0402] For example, consider a scenario where brightness is considered as the object environment, and the optimal observation method for the camera is inferred based on image data obtained from that camera. In this case, the object data (61, 225) is image data. The number of parameters in machine learning model 40 depends on the number of pixels in the image data and the number of states the camera can take, and could be on the order of thousands. On the other hand, the number of parameters in computational model 51 depends on the dimension of brightness and the number of trained machine learning models 45, but even if dozens of trained machine learning models 45 are used in the inference of the optimal observation method, it is at most on the order of tens to hundreds.

[0403] Therefore, generally speaking, the number of parameters in the computational model 51, which determines the value of the integration parameter P, should be less than the number of parameters in the machine learning model 40, which directly executes the prescribed inference. For example, when both the computational model 51 and the machine learning model 40 employ neural networks and supervised learning is used as the machine learning method, the size of the neural network in the computational model 51 can be smaller than that in the machine learning model 40. Furthermore, for example, when reinforcement learning is used as the machine learning method, the number of parameters in the environment for training the computational model 51 should be less than that in the scenario where the machine learning model 40 is trained. Since the cost of machine learning depends on the number of parameters, the cost of constructing the learned computational model 52 through machine learning can be suppressed to be lower than the cost of constructing a new learned machine learning model 45 suitable for the object environment. Therefore, according to the first method, the accuracy of the prescribed inference can be improved while suppressing the cost incurred in constructing an inference model that can appropriately execute the prescribed inference in a new environment.

[0404] (2) Second method

[0405] Figure 11AAn example is illustrated below illustrating a scenario where the values ​​of each integration parameter P are determined using the second method. In the second method, the control unit 11 determines the values ​​of each integration parameter P by comparing the object environment in which the prescribed inference is performed with the learning environment of the local learning data 30 used in the derivation of each inference model. Specifically, the control unit 11 calculates the fitness of the environment data 63 with each set of learning environment data 35, and determines the value of the integration parameter P for each corresponding inference model based on the calculated fitness.

[0406] Figure 11B This is a flowchart illustrating an example of the processing procedure for a subroutine that determines the values ​​of each integration parameter P using the second method. When using the second method, step S211 includes the processing steps S311 to S313. However, the processing described below is merely an example, and each process can be modified as much as possible. Furthermore, for the processing described below, steps can be appropriately omitted, substituted, or added according to the implementation method.

[0407] In step S311, the control unit 11 acquires learning environment data 35 related to the environment in which the local learning data 30 is used in the derivation of each inference model. As described above, in this embodiment, the learning environment data 35 of each learned machine learning model 45 can be provided from each local learning device 2 to the inference device 1 at any time. The control unit 11 can acquire each set of learning environment data 35 as appropriate. When each set of learning environment data 35 is acquired, the control unit 11 causes the process to proceed to the next step S312.

[0408] In step S312, the control unit 11 calculates the fitness of each set of learning environment data 35 and environment data 63. The form of fitness is not particularly limited, as long as the similarity between the target environment and the learning environment can be evaluated based on the calculated fitness, and can be appropriately determined according to the implementation method. For example, the control unit 11 can also calculate the consistency between the learning environment data 35 and environment data 63 as fitness. Alternatively, the control unit 11 can calculate the distance (norm) between the learning environment data 35 and environment data 63 and calculate fitness based on the calculated distance. In this case, the control unit 11 can calculate a smaller fitness value for a larger calculated distance and a larger fitness value for a smaller calculated distance. The correspondence between distance and fitness can be expressed through a data table, function, or rule. This correspondence can be manually set by the operator, or automatically derived, for example, through known methods such as fitting. Alternatively, the control unit 11 can calculate fitness using the reciprocal of the calculated distance. Fitness can be expressed using a predetermined number of levels, or it can be expressed using continuous values. Once each fitness level is calculated, the control unit 11 causes the processing to proceed to the next step S313.

[0409] In step S313, the control unit 11 determines the value of each integration parameter P for each corresponding inference model based on the calculated fitness scores. It is envisioned that the inference results derived from the inference model obtained from local learning data 30 obtained from a learning environment similar to the target environment have high reliability even in that target environment. Conversely, it is envisioned that the inference results derived from the inference model obtained from local learning data 30 obtained from a learning environment very different from the target environment have low reliability in that target environment. Therefore, the control unit 11 may calculate a larger value for the integration parameter P as the fitness score increases, and a smaller value for the integration parameter P as the fitness score decreases. The correspondence between the fitness score and the value of the integration parameter P can be represented by a data table, function, or rule. This correspondence can be manually set by an operator, or it can be automatically derived, for example, by a known method such as fitting. Alternatively, the control unit 11 may directly use the calculated fitness scores as the values ​​of each integration parameter P.

[0410] Therefore, the control unit 11 can determine the value of each integration parameter P for each inference model based on the calculated fitness. The control unit 11 can store the information representing the result of determining the value of each integration parameter P as at least a part of the integration rule data 57, or it can store it separately from the integration rule data 57. When the value of each integration parameter P is determined, the control unit 11 ends the processing of the subroutine related to step S211. In this embodiment, after determining the value of each integration parameter P, the control unit 11 integrates the inference results of each learned machine learning model 45 by executing step S212 and subsequent processing.

[0411] Unlike the first method, the second method eliminates the need for generating the learned computational model 52. Furthermore, the values ​​of each integration parameter P can be appropriately determined through a simple process such as comparing the object environment with the learning environment. Therefore, according to the second method, the cost of constructing an inference model capable of appropriately performing the prescribed inferences in a new environment can be reduced through a simpler approach.

[0412] (3) Third method

[0413] Figure 12A An example is illustrated below, illustrating a scenario where the values ​​of each integration parameter P are determined using a third method. In this third method, the control unit 11 receives the operator's specification of the value of the integration parameter P via the input device 15 and sets the value of the integration parameter P of the object to the specified value. Figure 12A An example of a designated receiving screen used to receive the value of the integration parameter P is shown schematically.

[0414] Figure 12BThis is a flowchart illustrating an example of the processing procedure for setting the value of the integration parameter P using a third method. The control unit 11 operates as the parameter setting unit 119, executing the following steps S321 to S323. However, the processing procedure described below is merely an example, and each procedure can be modified as much as possible. Furthermore, for the processing procedure described below, steps can be appropriately omitted, substituted, or added according to the implementation method.

[0415] In step S321, the control unit 11 outputs a receiving screen for receiving input of the value of the integrated parameter P for the inference model. In step S322, the control unit 11 receives, via the receiving screen, a specification of the value of at least one of the multiple integrated parameters P.

[0416] In this embodiment, the control unit 11 will Figure 12A The received screen is shown in the example output to output device 16. The output destination of the received screen is not limited to output device 16. For example, the received screen can be output to the output device of another computer. Figure 12A The example receiving screen includes a display bar 161, a selection bar 162, an input bar 163, and an OK button 165.

[0417] Information about each inference model is displayed in display bar 161. For example, the content of the learning environment data 35 can also be displayed in display bar 161. Additionally, if the aforementioned fitness level has been calculated, the calculated fitness level can also be displayed in display bar 161. A selection of whether to use the model in the inference process is received in selection bar 162. In this embodiment, the control unit 11 determines the learned machine learning model 45 to be used in the inference process based on the selection in selection bar 162.

[0418] The input field 163 receives the specified value of the integration parameter P for the object's inference model. The operator operates the input device 15 and can specify the corresponding integration parameter P by entering the desired value in the input field 163 for the object's inference model. The value of the integration parameter P can be specified by a predetermined number of levels or by consecutive values. It should be noted that by entering an invalid value (e.g., 0) in the input field 163, it can be specified that the object's inference model is not used. In this case, the selection field 162 can be omitted.

[0419] The OK button 165 is used to end the specification of the value of the integrated parameter P. After the specification of the value of the integrated parameter P is completed, the operator operates the input device 15 to operate the OK button 165. Based on the operation of the OK button 165, the control unit 11 ends the processing of step S322 and proceeds to the next step S323.

[0420] In step S323, the control unit 11 sets the value of the integration parameter P of the object to a specified value. The control unit 11 may save information related to the specified setting as at least a part of the integration rule data 57, or it may save it separately from the integration rule data 57. When the setting of the value of the integration parameter P of the object is completed, the control unit 11 ends the processing of the subroutine related to the setting of the value of the integration parameter P. In this embodiment, after setting the value of the integration parameter P, the control unit 11 integrates the inference results of each learned machine learning model 45 by executing step S212 and subsequent processing. In step S212, the control unit 11 uses the set value to weight the inference results of the learned machine learning model 45 corresponding to the integration parameter P of the object.

[0421] According to the third method, at least a portion of the values ​​of each integration parameter P applied to the inference results of each learned machine learning model 45 can be determined by the operator's specification. Therefore, the operator's intention (e.g., emphasizing the inference results of a specific learned machine learning model 45) can be reflected in the prescribed inference performed in the object environment.

[0422] (4) Summary

[0423] In this embodiment, the control unit 11 can determine the value of each integration parameter P in step S211 by employing at least one of the three methods described above. The three methods can be appropriately employed. For example, after determining the value of each integration parameter P using the first or second method, the control unit 11 may also correct the value of at least one of the multiple integration parameters P using the third method.

[0424] <Grouping of Inference Models>

[0425] Next, use Figure 13A The grouping of each inference model is explained. Figure 13A An example of a scenario where inference models are grouped is illustrated. In this embodiment, the learned machine learning model 45, which is an example of an inference model, is generated by each local learning device 2. Therefore, one of the learned machine learning models 45 may be generated for completely different purposes to perform different inferences from other learned machine learning models 45. In this case, it is difficult to integrate the inference results of one learned machine learning model 45 with the inference results of other learned machine learning models 45.

[0426] Therefore, in this embodiment, the control unit 11 can also function as a grouping unit 1110, assigning each inference model (in this embodiment, each learned machine learning model 45) to at least one of the multiple groups. Each group can be appropriately configured according to the type of inference, purpose, target environment, etc. Figure 13A This example illustrates a scenario where each learned machine learning model 45 is assigned to two groups, A and B. The control unit 11 stores the assignment results for each learned machine learning model 45 in assignment information 125. Assignment information 125 can be stored, for example, in a designated storage area. The designated storage area can be RAM within the control unit 11, storage unit 12, external storage device, storage media, or a combination thereof.

[0427] Correspondingly, integration rule 5 is prepared for each group. In step S203 above, the control unit 11 can also determine the group used in the object environment among the multiple groups set, and perform the process of obtaining the above inference results for the machine learning model 45 that has completed learning and belongs to the determined group. Then, in step S204 above, the control unit 11 can also integrate the inference results of the machine learning model 45 that has completed learning and belongs to the determined group according to the corresponding integration rule 5. Thus, integration processing can be performed for each group according to the purpose.

[0428] It should be noted that the grouping method is not particularly limited and can be appropriately determined according to the implementation method. In this embodiment, the control unit 11 can assign each inference model to at least one of the multiple groups using either of the following two methods.

[0429] (1) First grouping method

[0430] First, use Figure 13B An example of the first grouping method is illustrated. Figure 13B This is a flowchart illustrating an example of a processing procedure related to the grouping of inference models using a first grouping method. In the first grouping method, the control unit 11 assigns each inference model to at least one of a plurality of groups by selecting a desired group from a list of groups.

[0431] It should be noted that when the first grouping method is used to assign each inference model to a group, assigning each inference model to at least one of the multiple groups consists of the processes of steps S411 and S412. However, the process described below is only an example, and each process can be modified as much as possible. In addition, for the process described below, steps can be omitted, substituted, or added appropriately according to the implementation method.

[0432] In step S411, the control unit 11 receives a selection of one or more groups assigned to each inference model from a list showing multiple groups. For example, the control unit 11 may also output the list showing multiple groups to the output device 16. Correspondingly, the operator may also operate the input device 15 to select one or more groups assigned to each inference model from the list. Thus, it is possible to receive a selection of one or more groups assigned to each inference model from a list showing multiple groups.

[0433] It should be noted that group selection can also be implemented without using the inference device 1. For example, group selection can also be implemented by each local learning device 2. As a specific example, the control unit 11 can also distribute a list showing multiple groups to each local learning device 2. Thus, the control unit 11 can cause each local learning device 2 to select one or more groups from the multiple groups shown in the list.

[0434] Furthermore, the groups included in the list can be set based on attributes such as the local learning data 30, the local learning device 2, and the user of the local learning device 2. For example, in the case of inferring the optimal observation method of the sensor, consider a scenario where the sensor is used to monitor the status of a factory production line. In this case, groups can also be set based on attributes such as production line number, factory name, and company name. New groups can be added to the list based on operator actions or requests from each local learning device 2.

[0435] In step S412, based on the response to step S411, the control unit 11 assigns each inference model to one or more selected groups. Once the assignment to one or more groups is complete, the control unit 11 terminates the series of processes related to group assignment using the first grouping method. According to this first grouping method, the control unit 11 can group each inference model using a simple method.

[0436] (2) Second grouping method

[0437] Next, use Figure 13C An example of the second grouping method is illustrated. Figure 13C This is a flowchart illustrating an example of the processing procedure related to the grouping of each inference model using a second grouping method. In the second grouping method, the control unit 11 assigns each inference model to an appropriate group based on the attributes of the local learning data 30.

[0438] It should be noted that when the second grouping method is used to assign each inference model to a group, assigning each inference model to at least one of the multiple groups consists of the processes of steps S421 to S423. However, the process described below is only an example, and each process can be modified as much as possible. In addition, for the process described below, steps can be omitted, substituted, or added appropriately according to the implementation method.

[0439] (Step S421)

[0440] In step S421, the control unit 11 acquires attribute data related to the local learning data 30 from each local learning device 2. The method for acquiring the attribute data can be the same as the method for acquiring the learning environment data 35 described above. The control unit 11 can appropriately acquire attribute data from each local learning device 2.

[0441] The attribute data may also include all information related to the local learning data 30. For example, the attribute data may include information indicating the category of the data included in the local learning data 30, information indicating the features exhibited in the data, information indicating the purpose of using the data, etc. Additionally, the attribute data may also include the learning environment data 35. The attribute data may be generated during at least one of the following steps: acquiring the local learning data 30 in step S101 and acquiring the learning environment data 35 in step S103. When the attribute data is acquired, the control unit 11 causes the processing to proceed to the next step, S422.

[0442] (Steps S422 and S423)

[0443] In step S422, the control unit 11 clusters the attribute data acquired from each local learning device 2. The clustering method is not particularly limited and can be appropriately selected depending on the implementation method. Known methods such as k-means clustering can be used for clustering.

[0444] In step S423, the control unit 11 assigns each inference model to at least one of the multiple groups based on the clustering results. For example, the control unit 11 assigns inference models whose acquired attribute data belongs to the same class to the same group. In this case, each group can be set according to the class of the attribute data. Alternatively, the control unit 11 can also assign each inference model to two or more groups based on the clustering results.

[0445] Once the group assignment based on the clustering results is complete, the control unit 11 terminates a series of processes related to the group assignment performed via the second grouping method. According to this second grouping method, the control unit 11 is able to assign each inference model to an appropriate group based on the attributes of the local learning data 30.

[0446] By employing at least one of the two methods described above, the control unit 11 can appropriately group the various inference models. However, the grouping method is not limited to these examples and can be appropriately determined according to the implementation method.

[0447] [feature]

[0448] As described above, the inference device 1 of this embodiment, in order to perform the prescribed inference in the target environment, does not construct a new inference model, but flexibly applies multiple learned machine learning models 45 that have already been constructed in the processing of steps S203 and S204. In the integration processing of steps S211 to S213, the inference device 1 of this embodiment adjusts the degree of emphasis on the inference of each learned machine learning model 45 according to the target environment shown in the environment data 63. As a result, the inference device 1 of this embodiment can customize the insights related to the prescribed inference obtained in various environments to be suitable for new environments. As a result, the prescribed inference can be appropriately performed in the target environment based on the customized insights, in other words, based on the weighted inference results of integrating each learned machine learning model 45.

[0449] Therefore, according to this embodiment, by flexibly applying multiple learned machine learning models 45 derived from local learning data 30 obtained in different environments, a new inference model capable of appropriately performing prescribed inferences in the target environment can be constructed. Thus, the cumbersome task of collecting new learning data in the target environment and deriving new inference models, particularly generating new learned machine learning models, can be eliminated. Therefore, according to this embodiment, the cost of constructing an inference model capable of appropriately performing prescribed inferences in a new environment can be reduced.

[0450] §4 Variations

[0451] The embodiments of the present invention have been described in detail above, and the description so far is merely illustrative in all respects. Various modifications or variations can be made without departing from the scope of the present invention. For example, the following changes can be made. It should be noted that, below, the same reference numerals are used for the same constituent elements as in the above embodiments, and descriptions of aspects identical to those in the above embodiments are appropriately omitted. The following variations can be appropriately combined.

[0452] <4.1>

[0453] The inference device 1 and each local learning device 2 described in the above embodiments can be applied to all scenarios in which an inference is performed through an inference model. Scenarios in which an inference is performed include, for example, inferring the optimal observation method for a sensor, predicting the movement path of a moving object, inferring a conversational strategy suitable for a user, and inferring action instructions for a robot device suitable for a task. Four variations limiting the application scenarios are illustrated below.

[0454] (A) Scenario for inferring the optimal observation method for the sensor

[0455] Figure 14 This example illustrates one application scenario of the inspection system 100A involved in the first modification. This modification is an example of applying the above-described embodiment to a scenario where the optimal observation method for a sensor is inferred. Figure 14 As shown, the inspection system 100A according to this modification includes an inspection device 1A and multiple local learning devices 2A. Similar to the embodiments described above, the inspection device 1A and each local learning device 2A can be interconnected via a network.

[0456] The checking device 1A corresponds to the inference device 1 described above. Each local learning device 2A corresponds to each local learning device 2 described above. Except for the limitation on the processed data and the content of the inference, the checking device 1A can be configured in the same way as the inference device 1, and each local learning device 2A can be configured in the same way as each local learning device 2. The hardware and software configurations of the checking device 1A can be the same as those of the inference device 1. The hardware and software configurations of each local learning device 2A can be the same as those of each local learning device 2.

[0457] In this variation, sensor SA1 is connected to inspection device 1A and each local learning device 2A. The prescribed inference involved in this variation is a method for inferring a change from the current observation state of the properties of the object RA observed by sensor SA1 to an appropriate observation state for obtaining observation data suitable for prescribed conditions through sensor SA1. Sensor SA1 can be, for example, a camera, microphone, life sensor, etc. The camera can be, for example, a general digital camera configured to acquire RGB images, a depth camera configured to acquire depth images, an infrared camera configured to image infrared quantities, etc. The object RA can be, for example, a product, plant, person, etc.

[0458] The specified conditions can be appropriately determined based on the observation purpose of the sensor SA1. For example, consider the case where observation data is obtained by the sensor SA1 to evaluate the quality of an object RA. In this case, the specified conditions can be determined based on the performance of inferences related to the quality evaluation of the observation data obtained by the sensor SA1. The quality evaluation of the object RA can be, for example, product inspection (e.g., defect detection), inspection of the growth status of plants (e.g., crops), inspection of the health status of people, etc. As a specific example, in the case of product inspection, the specified conditions can be determined based on the performance of the target inspection of the observation data obtained by the sensor SA1, such as whether defect detection can be performed based on the observation data obtained by the sensor SA1, and whether the accuracy of defect detection meets the benchmark.

[0459] In this variation, each local learning device 2A generates a learned machine learning model 45A that has acquired the ability to infer a change method for sensor SA1 to an appropriate observation state. Conversely, the inspection device 1A in this variation uses each learned machine learning model 45A derived from local learning data 30A obtained by each local learning device 2A in different environments to infer the optimal observation method for sensor SA1 of the object in the object environment. In this variation, the sensor SA1 of the object for which the optimal observation method is inferred is the sensor SA1 connected to the inspection device 1A. However, the sensor SA1 of the object is not limited to this; the inspection device 1A may also infer the optimal observation method for sensor SA1 connected to other computers.

[0460] The inspection device 1A involved in this modification acquires object data 61A, which is the object of a predetermined inference, and environmental data 63A related to the object environment in which the predetermined inference is performed. The object data 61A, which is the object of the predetermined inference, is data related to the current observation state of the object's sensor SA1. On the other hand, the environmental data 63A, which is related to the object environment in which the predetermined inference is performed, is data related to the object observation environment in which the object's sensor SA1 observes the properties of the object RA.

[0461] In this modified example, the inspection device 1A is further connected to other sensors SA2. These other sensors SA2 differ from the sensor SA1 used to infer the optimal observation method for the object; they are used to observe the object's observation environment. Therefore, the inspection device 1A can acquire environmental data 63A from the other sensors SA2. Considering factors such as brightness, temperature, and humidity, the other sensors SA2 can be, for example, a camera, a photometer, an illuminance meter, a thermometer, a hygrometer, etc.

[0462] The inspection device 1A in this modification provides the acquired object data 61A to each learned machine learning model 45A, enabling each learned machine learning model 45A to infer a method for changing the sensor SA1 to an appropriate observation state. Thus, the inspection device 1A in this modification obtains the inference results of each learned machine learning model 45A regarding the method for changing the sensor SA1 to an appropriate observation state. Then, the inspection device 1A in this modification integrates the inference results of each learned machine learning model 45A according to the integration rule 5A.

[0463] Similar to the above-described embodiment, the integration rule 5A includes multiple integration parameters PA that specify the degree of importance given to the inference results of each learned machine learning model 45A in the object environment. The inspection device 1A of this variant determines the values ​​of each integration parameter PA based on the environmental data 63A. At this time, in addition to the environmental data 63A, the object data 61A may also be considered. Next, the inspection device 1A of this variant uses the determined values ​​of each integration parameter PA to weight the inference results of the corresponding learned machine learning models 45A. Then, the inspection device 1A of this variant integrates the weighted inference results of each learned machine learning model 45A.

[0464] Therefore, the inspection device 1A according to this modification can generate a result of inferring a change method for changing the current observation state of the sensor SA1 of the object to an appropriate observation state in the object environment. The change method for inferring the change to an appropriate observation state may include directly inferring the change method, or indirectly inferring the change method for guiding to the appropriate observation state by inferring the appropriate observation state and calculating the difference between the inferred appropriate observation state and the current observation state.

[0465] It should be noted that, Figure 14 The symbols illustratively illustrate a scenario of product image inspection. The object RA can be a product manufactured on a production line, and the sensor SA1 can be a camera. The observation data obtained by the sensor SA1 can be image data. Considering brightness in the image inspection scenario, the other sensor SA2 can also be a photometer or illuminance meter. The specified conditions can be conditions related to capturing the object RA (product) in image data in a manner suitable for inspecting the attributes of the object RA (product) (e.g., presence or absence of defects, type of defects).

[0466] In this modified example, the inspection device 1A is further connected to the gimbal device IN1 and the illumination device IN2. The sensor S1 is mounted on the gimbal device IN1. The gimbal device IN1 includes a base portion In11, a first joint portion In12, and a second joint portion In13. Each joint portion (In12, In13) may include a drive device such as a servo motor. The first joint portion In12 is connected to the base portion In11, causing the front end portion to rotate about the axis of the base. The second joint portion In13 is connected to the first joint portion In12, causing the front end portion to rotate in a forward-backward direction. The gimbal device IN1 is configured such that the orientation and configuration of the sensor S1 can be changed under computer control by incorporating these components. Furthermore, the illumination device IN2 can be configured to change the brightness of the observation environment by adjusting the output light intensity under computer control.

[0467] By using the gimbal device IN1 and the illumination device IN2, the setting angle of sensor SA1 and the brightness around the object RA in the observation state of sensor SA1 can be changed. The gimbal device IN1 and the illumination device IN2 are examples of the "intervention device" of this invention. However, the intervention device is not limited to just these examples, as long as it is configured to change the observation state based on the observation state of the sensor SA1 of the intervention object. In cases of intervention temperature and humidity, air conditioning units, humidifiers, heaters, etc., can be used as intervention devices. Furthermore, in cases where the position or posture of at least one of the intervention object RA and sensor SA1 is important, conveyor devices, robotic arms, etc., can be used as intervention devices.

[0468] <Generation of the Machine Learning Model After Learning>

[0469] In this variation, each local learning device 2A, like the local learning devices 2 described above, generates a learned machine learning model 45A by executing steps S101 to S104. Specifically, in step S101, each local learning device 2A acquires local learning data 30A. Then, in step S102, each local learning device 2A uses the acquired local learning data 30A to perform machine learning on the machine learning model 40A. The structure and method of the machine learning model 40A can be the same as those of the machine learning model 40 described above.

[0470] As an example, the machine learning method can use either supervised learning (first example) or reinforcement learning (third example) as described above. In the case of supervised learning, the local learning data 30A consists of multiple learning datasets, each including a combination of training data and forward solution data. The training data is the same type as the object data 61A described above and can consist of data related to the current observation state of sensor SA1. The forward solution data can consist of data representing the result (forward solution) of the inference from the training data, i.e., the change method from the current observation state to an appropriate observation state. Each learning dataset can be generated using the same method as in the above embodiment. Each local learning device 2A performs supervised learning of the machine learning model 40A using the acquired local learning data 30A. The supervised learning method can be the same as in the above embodiment.

[0471] Furthermore, in the case of reinforcement learning, the machine learning model 40A can be value-based, policy-based, or both. The state of the observed object can be the observation state of sensor SA1, and the action performed by the agent can be a change in the observation state of sensor SA1. The reward function can also be manually set by an operator, etc. Alternatively, the reward function can be set to provide an immediate reward based on the appropriateness of the inference result of the machine learning model 40A. In this case, similarly to the above-described embodiment, the appropriateness can also be manually provided by an operator, etc. Alternatively, the appropriateness can also be evaluated using a decision maker according to a prescribed benchmark. The decision maker can be configured to perform target inference processing on the observation data obtained from sensor SA1. For example, in the scenario of product image inspection, the decision maker can be configured to detect product defects based on image data. Correspondingly, the reward function can be set to provide a positive immediate reward based on the trial results of multiple inference processes, if the accuracy of the inference process is above a threshold, and a negative immediate reward if the accuracy of the inference process is below an acceptable value (e.g., the defect cannot be detected). Alternatively, the reward function can also be inferred from example data representing expert demonstrations through inverse reinforcement learning. In this variation, the example data may consist, for example, data representing the method of changing sensor SA1 obtained through the operation of a skilled operator. Each local learning device 2A adjusts the values ​​of the operational parameters of the machine learning model 40A to maximize the value (expected value) obtained in an appropriately set learning environment. The reinforcement learning method can be the same as in the above-described embodiment.

[0472] Each local learning device 2A can perform machine learning of the machine learning model 40A using any of the methods described above. Thus, each local learning device 2A can generate a learned machine learning model 45A that has acquired the ability to infer a change method from the current observation state of the properties of the object RA observed by the sensor SA1 to an appropriate observation state for obtaining observation data suitable for specified conditions through the sensor SA1.

[0473] In step S103, each local learning device 2A acquires learning environment data 35A related to the environment in which the local learning data 30A is obtained. The learning environment data 35A is the same type of data as the environment data 63A obtained by the inspection device 1A. The method for acquiring the learning environment data 35A can be the same as the method for acquiring the environment data 63A. For example, each local learning device 2A can be further connected to a sensor of the same type as other sensors SA2, and can also acquire the learning environment data 35A from that sensor. In step S104, each local learning device 2A generates information related to the generated, learned machine learning model 45A as learning result data 47A. Then, each local learning device 2A associates the learning result data 47A with the learning environment data 35A and stores it in a designated storage area.

[0474] In this variation, local learning data 30A can be acquired under different environments among the various local learning devices 2A. Then, a fully learned machine learning model 45A can be generated based on the acquired local learning data 30A. As a result, multiple fully learned machine learning models 45A derived from the local learning data 30A acquired under different environments can be obtained in a manner that allows for the inference of the optimal observation method for sensor SA1.

[0475] <Hardware Composition of the Inspection Device>

[0476] Figure 15 An example of the hardware configuration of the inspection device 1A involved in this modification is illustrated schematically. For example... Figure 15As shown, the inspection device 1A involved in this variation, like the inferred device 1 described above, is a computer electrically connected to the control unit 11, storage unit 12, communication interface 13, external interface 14, input device 15, output device 16, and driver 17. The inspection device 1A is connected to sensor SA1, other sensors SA2, pan / tilt device IN1, and lighting device IN2 via the external interface 14. However, the hardware configuration of the inspection device 1A is not limited to this example. Regarding the specific hardware configuration of the inspection device 1A, constituent elements can be appropriately omitted, substituted, or added according to the implementation method. Besides being an information processing device designed specifically for the services provided, the inspection device 1A can also be a general-purpose server device, a general-purpose PC, a PLC (programmable logic controller), etc.

[0477] In this modified example, the storage unit 12 of the inspection device 1A stores various information such as inspection program 81A, integrated rule data 57A, learning result data 47A, learning data 59A, and learning environment data 35A. The inspection program 81A, integrated rule data 57A, learning result data 47A, learning data 59A, and learning environment data 35A correspond to the inference program 81, integrated rule data 57, learning result data 47, learning data 59, and learning environment data 35A involved in the above-described embodiment. At least any one of the inspection program 81A, integrated rule data 57A, learning result data 47A, learning data 59A, and learning environment data 35A may also be stored in the storage medium 91. Furthermore, the inspection device 1A may also obtain at least any one of the inspection program 81A, integrated rule data 57A, learning result data 47A, learning data 59A, and learning environment data 35A from the storage medium 91.

[0478] <Software Structure of the Inspection Device>

[0479] Figure 16A and Figure 16B An example of the software configuration of the inspection device 1A according to this modification is illustrated schematically. Similar to the embodiment described above, the software configuration of the inspection device 1A is implemented by the control unit 11 executing the inspection program 81A. For example... Figure 16A and Figure 16B As shown, except for the limitation on the processed data and the content of the inference, the software configuration of the inspection device 1A is the same as that of the inference device 1. Therefore, the inspection device 1A performs a series of processes related to the inference in the same way as the inference device 1.

[0480] (Step S201)

[0481] That is, such as Figure 16AAs shown, in step S201, the control unit 11 of the inspection device 1A acquires object data 61A. Object data 61A can be any data related to the current observation state of the object's sensor SA1, and its content is not particularly limited, but can be appropriately selected according to the implementation. Object data 61A may include, for example, data representing attributes related to the setting state of the object sensor SA1, data representing attributes related to the operation settings of the object sensor SA1, and observation data obtained from the object sensor SA1. Attributes related to the setting state of the object sensor SA1 may include, for example, the setting angle of the sensor SA1, the distance between the sensor SA1 and the observed object (object RA), and observation attributes that affect the observation of the sensor SA1 (e.g., brightness, temperature, humidity, etc.). Attributes related to the operation settings of the object sensor SA1 may include, for example, the setting value of the sensor SA1's measurement range, the setting value of the measurement range's resolution, and the setting value of the sampling frequency. As a specific example, if the sensor SA1 is a camera, the attributes related to the operation settings may include aperture value, shutter speed, zoom ratio, etc.

[0482] The method for acquiring object data 61A is not particularly limited and can be appropriately selected according to the implementation method. For example, object data 61A can be acquired from the object sensor SA1 itself. Alternatively, object data 61A can be acquired from other sensors that observe environmental properties (e.g., other sensor SA2). In this modified example, as interventional devices, the pan-tilt device IN1 and the illumination device IN2 are connected to the inspection device 1A. Object data 61A can also be acquired from at least one of the interventional devices themselves and sensors (not shown) that observe the state of each interventional device. Alternatively, object data 61A can be acquired through input from an operator or the like. The control unit 11 can acquire object data 61A directly from each device or indirectly via another computer.

[0483] (Step S202)

[0484] In step S202, the control unit 11 acquires environmental data 63A. Environmental data 63A can be any data related to the object observation environment of the sensor SA1 observing the properties of the object RA; its content is not particularly limited and can be appropriately selected according to the implementation method. Environmental data 63A may include, for example, data representing attributes related to the specifications (or performance) of the sensor SA1, data representing the attributes of the observed object (object RA), and environmental attributes affecting the observation of the sensor SA1. Attributes related to the specifications (or performance) of the sensor SA1 may include, for example, the sensor SA1's sensitivity limit, dynamic range, settable range of spatial resolution, and settable range of sampling frequency. Attributes of the observed object (object RA) may include, for example, the type of observed object and the purpose of observation. The type of observed object may be, for example, the type of product / workpiece, the type of plant, etc. The purpose of observation may be, for example, defect detection, quality inspection, etc.

[0485] The method for acquiring environmental data 63A is not particularly limited and can be appropriately selected depending on the implementation method. For example, environmental data 63A can also be acquired through input from an operator or the like. Alternatively, environmental data 63A can be acquired from the object sensor SA1 itself. Alternatively, environmental data 63A can also be acquired from observation data obtained by the object sensor SA1. Alternatively, environmental data 63A can also be acquired from other sensors that observe environmental properties (e.g., other sensor SA2). The control unit 11 can acquire environmental data 63A directly from each device or indirectly via another computer.

[0486] (Step S203)

[0487] In step S203, the control unit 11 sets up each learned machine learning model 45A by referring to each set of learning result data 47A. Next, the control unit 11 provides object data 61A to each learned machine learning model 45A and performs computational processing on each learned machine learning model 45A. Thus, the control unit 11 obtains the inference results of each learned machine learning model 45A regarding the change method for the object sensor SA1 to an appropriate observation state, and uses this as the output of each learned machine learning model 45A.

[0488] (Step S204)

[0489] In step S204, the control unit 11 sets the integration rule 5A with reference to the integration rule data 57A. Then, the control unit 11 integrates the inference results of each learned machine learning model 45A according to the integration rule 5A. Specifically, in step S211, the control unit 11 determines the value of each integration parameter PA based on the environmental data 63A. At this time, in addition to the environmental data 63A, the object data 61A may also be considered. The method for determining the value of each integration parameter PA can be any of the first to third methods described below.

[0490] (1) First method

[0491] like Figure 16B As shown, when using the first method, the control unit 11 acquires learning data 59A. Then, the control unit 11 uses the learning data 59A to perform machine learning on the computational model 51A. The configuration of the computational model 51A and the machine learning method can be the same as those of the computational model 51 described above. The configuration of the computational model 51A and the machine learning method can adopt either of the two examples described above.

[0492] In the first example, learning data 59A, like learning data 59 described above, can be composed of multiple learning datasets, each including combinations of training environment data, training object data, and forward solution data. The training environment data is the same type of data as environment data 63A. The training object data is the same type of data as object data 61A. The forward solution data can be composed of data representing the inference result (forward solution) from the training object data, i.e., the change method from the current observation state to an appropriate observation state in the object environment. Each learning dataset can be generated using the same method as in the above embodiment. The control unit 11 uses the acquired learning data 59A to perform machine learning on the computational model 51A. The machine learning method can be the same as in the first example described above.

[0493] In the second example, the computational model 51A can be value-based, policy-based, or both. Similar to the above implementation, the state of the observed object corresponds to the environmental data 63A and the object data 61A. The action performed by the agent can be a change in the observed state of sensor SA1. The agent obtains the inference results of each learned machine learning model 45A through the above series of processes, and by integrating the obtained inference results, it can generate an inference result for the object environment. The agent can also determine the action to be taken based on the generated inference result.

[0494] The reward function can also be manually set by the operator. Alternatively, the reward function can be set to provide an immediate reward based on the appropriateness of the inference result generated through the above series of processes. The appropriateness can be provided manually by the operator. Alternatively, the appropriateness can also be evaluated using a decision maker according to a prescribed benchmark. The decision maker can be the same as the decision maker in the reinforcement learning of the machine learning model 40A described above. Alternatively, the reward function can also be inferred through inverse reinforcement learning based on example data representing expert demonstrations. This example data can be the same as the example data used in setting the reward function in the reinforcement learning of the machine learning model 40A described above. The control unit 11 adjusts the values ​​of the operation parameters of the operation model 51A to maximize the value (expected value) obtained in an appropriately set learning environment. The reinforcement learning method can be the same as the above-described implementation method.

[0495] The control unit 11 can perform machine learning on the computational model 51A using any of the methods described above. Thus, the control unit 11 can generate a learned computational model 52A that has acquired the ability to infer the values ​​of each integrated parameter PA based on the environmental data 63A (and the object data 61A) to determine the optimal observation method for the sensor SA1 under the object observation environment. The control unit 11 can also store information related to the generated learned computational model 52A in a designated storage area. The information related to the learned computational model 52A can be stored as at least a part of the integrated rule data 57A, or it can be stored separately from the integrated rule data 57A.

[0496] In the first method, the control unit 11 determines the values ​​of each integration parameter PA using the learned computational model 52A generated above. That is, in step S211, the control unit 11 provides environmental data 63A to the learned computational model 52A. At this time, the control unit 11 can further provide object data 61A to the learned computational model 52A. Then, the control unit 11 performs computational processing on the learned computational model 52A. As a result, the control unit 11 can obtain the values ​​of each integration parameter PA for each learned machine learning model 45A as the output of the learned computational model 52A.

[0497] (2) Second method

[0498] In the second method, the control unit 11 determines the value of each integration parameter PA based on a comparison between the object environment and each learning environment. Specifically, the control unit 11 acquires the learning environment data 35A for each learned machine learning model 45A. Next, the control unit 11 calculates the fitness of each set of learning environment data 35A and environment data 63A. The form and representation of the fitness can be the same as in the above embodiment. In step S211 above, the control unit 11 can also determine the value of each integration parameter PA for each corresponding learned machine learning model 45A based on the calculated fitness. The method for determining the value of the integration parameter PA based on the fitness can be the same as in the above embodiment.

[0499] (3) Third method

[0500] In the third method, the control unit 11 receives the operator's specification of the value of the integration parameter PA and sets the value of the object's integration parameter PA to the specified value. The receiving screen can be the same as in the above embodiment. Figure 12A ).

[0501] Return to Figure 16A In step S211, the control unit 11 determines the value of each integration parameter PA by employing at least one of the three methods described above. In step S212, the control unit 11 uses the determined values ​​of each integration parameter PA to weight the inference results of the corresponding learned machine learning models 45A. If at least one integration parameter PA is set using the third method described above, in step S212, the control unit 11 uses the set value to weight the inference results of the learned machine learning models 45A corresponding to the integration parameter PA of the object. In step S213, the control unit 11 integrates the weighted inference results of each learned machine learning model 45A. The integration of the inference results can be performed using a weighted average or a weighted majority vote, similar to the above embodiment. This allows for the generation of a method for inferring a change from the current observation state of the object sensor SA1 to an appropriate observation state in the object environment.

[0502] (Step S205)

[0503] In step S205, the control unit 11 outputs information related to the generated inference result. Similar to the above embodiment, the output destination and the content of the output information can be appropriately determined according to the embodiment. The control unit 11 can directly output the inference result generated in step S204 to the output device 16, or it can perform certain information processing based on the generated inference result.

[0504] For example, the control unit 11 may also generate, based on the inference result generated in step S204, instruction information to instruct the user to change the observation state of the target sensor SA1 according to the inferred change to an appropriate observation state, as information related to the inference result. The control unit 11 may then output the generated instruction information to an output device. The output device to which the output is destined may be the output device 16 of the inspection device 1A, or it may be the output device of another computer. The other computer may be a computer located near the user, or it may be a terminal device held by the user. According to this output method, even users without specialized knowledge can optimize the observation state of the sensor SA1 according to the target environment.

[0505] Furthermore, consider a scenario where the inspection device 1A is connected to an intervention device that changes the observation state of the target sensor SA1. In this case, the control unit 11 can also generate instruction information related to the inference result, which instructs the intervention device to perform an action to change the observation state of the target sensor SA1 according to the inferred change method. Then, the control unit 11 can send the instruction information to the intervention device to perform an action to change the observation state of the target sensor SA1 according to the inferred change method. In this case, the control unit 11 can also directly control the action of the intervention device. Alternatively, if the intervention device has a control device, the control unit 11 can also send instruction information to the control device to indirectly control the action of the intervention device by causing the control device to perform the control of the intervention device's action. According to this output method, the observation state of the sensor SA1 can be automatically optimized.

[0506] In this modified example, the inspection device 1A is connected to a gimbal device IN1 and an illumination device IN2, which serve as intervention devices. The control unit 11 can also control the operation of at least one of the gimbal device IN1 and the illumination device IN2 based on the inference result generated in step S204, thereby changing the observation state of the object sensor SA1. This allows for the optimization of at least one of the setting angle of the object sensor SA1 and the brightness around the object RA.

[0507] It should be noted that, similar to the embodiments described above, the control unit 11 of the inspection device 1A can assign each learned machine learning model 45A to at least one of the multiple groups. Therefore, the control unit 11 can also perform integration processing on each group according to the purpose. The grouping method can be any of the two methods described above.

[0508] <Inferential processing of local learning devices>

[0509] Furthermore, in this modified example, each local learning device 2A, like the aforementioned local learning devices 2, can infer the optimal observation method for sensor SA1 using the learned machine learning model 45A by executing steps S111 to S113. The sensor SA1 for which the optimal observation method is inferred by each local learning device 2A is typically a sensor SA1 connected to each local learning device 2A itself. However, the sensor SA1 is not limited to this; each local learning device 2A can also infer the optimal observation method for sensors SA1 connected to other computers.

[0510] In step S111, each local learning device 2A acquires the object data used in the inference. The method for acquiring the object data can be the same as step S201 of the inspection device 1A described above. In step S112, each local learning device 2A provides the object data to the learned machine learning model 45A and performs computational processing on the learned machine learning model 45A. Thus, each local learning device 2A can acquire the result of inferring the change method from the current observation state of sensor SA1 to an appropriate observation state, as the output of the learned machine learning model 45A.

[0511] In step S113, each local learning device 2A outputs information related to the inference result. The output destination and the content of the output information can be appropriately determined according to the implementation method. Each local learning device 2A can directly output the inference result obtained in step S112 to the output device, or it can perform certain information processing based on the obtained inference result.

[0512] Furthermore, the processing in step S113 can be performed in the same manner as step S205 of the inspection device 1A described above. For example, each local learning device 2A can also generate instruction information to instruct the user to change the observation state of sensor SA1 according to the inferred change method to an appropriate observation state, and output the generated instruction information to an output device. In addition, each local learning device 2A can also be further connected to an intervention device such as a gimbal device IN1 or an illumination device IN2. In this case, each local learning device 2A can also generate instruction information to cause the intervention device to perform an action to change the observation state of sensor SA1 according to the inferred change method to an appropriate observation state. Then, each local learning device 2A can also send instruction information to the intervention device to cause the intervention device to perform an action to change the observation state of sensor SA1 according to the inferred change method.

[0513] <Features>

[0514] According to this variation, in scenarios where the properties of an object RA are observed by sensor SA1, the cost of constructing an inference model that can appropriately infer the optimal observation method of sensor SA1 in a new environment can be reduced. Furthermore, by utilizing another sensor SA2 in the observation of the observation state of sensor SA1, environmental data 63A that fully reflects the observation state of sensor SA1 can be acquired. Thus, the accuracy of the inference of the optimal observation method of object sensor SA1 by the inspection device 1A can be improved.

[0515] Furthermore, by processing the output of the inspection device 1A, the observation state of the object sensor SA1 can be optimized automatically or manually by the user. This allows for systematic optimization of the observation state of sensor SA1 in various environments without relying on manual standardization. Therefore, according to this variation, a method for observing the properties of an object RA for product defect detection, plant quality inspection, etc., can be adapted at low cost in various environments. Thus, high-quality inspection with suppressed deviations can be implemented in various environments.

[0516] It should be noted that this variation can be modified appropriately. For example, if environmental data 63A is acquired from a sensor other than SA2, SA2 can be omitted. If the observation state is changed without intervention, the pan-tilt unit IN1 and the illumination unit IN2 can be omitted. The inspection device 1A can be connected to each device (sensor SA1, other sensor SA2, pan-tilt unit IN1, and illumination unit IN2) via a communication interface. The inspection device 1A can also be configured to output both indication information and command information in step S205 above. Alternatively, either the indication information or the command information can be omitted.

[0517] (B) Scenario of predicting the movement path of a mobile object

[0518] Figure 17 This example illustrates one application scenario of the prediction system 100B involved in the second variation. This variation is an example of applying the above-described implementation to a scenario of predicting the movement path of a moving body. For example... Figure 17 As shown, the prediction system 100B involved in this modified example includes a prediction device 1B and multiple local learning devices 2B. Similar to the embodiments described above, the prediction device 1B and each local learning device 2B can be interconnected via a network.

[0519] The prediction device 1B corresponds to the inference device 1 described above. Each local learning device 2B corresponds to each local learning device 2 described above. Except for the limitation on the data processed and the content of the inference, the prediction device 1B can be configured in the same way as the inference device 1, and each local learning device 2B can be configured in the same way as each local learning device 2. The hardware and software configurations of the prediction device 1B can be the same as those of the inference device 1. The hardware and software configurations of each local learning device 2B can be the same as those of each local learning device 2.

[0520] The inference in this variation involves predicting the movement path of the moving object RB based on its state. The moving object RB can be any moving object, and its type is not particularly limited; it can be appropriately selected according to the implementation method. The moving object RB can be, for example, a living organism (human, etc.), or a machine (vehicle, etc.). In this variation, sensor SB1 is connected to prediction device 1B and each local learning device 2B. The state of the moving object RB is observed by sensor SB1. Sensor SB1 can be, for example, a camera, a beacon, etc. The camera can be, for example, a conventional digital camera configured to acquire RGB images, a depth camera configured to acquire depth images, or an infrared camera configured to image infrared quantities, etc.

[0521] In this variation, each local learning device 2B generates a learned machine learning model 45B that has acquired the ability to predict the movement path of the moving body RB based on its state. In contrast, the prediction device 1B in this variation uses each learned machine learning model 45B derived from local learning data 30B obtained by each local learning device 2B in different environments to predict the movement path of the moving body RB in the object environment.

[0522] The prediction device 1B in this modification acquires object data 61B, which is the object of a predetermined inference, and environmental data 63B, which is related to the object environment in which the predetermined inference is performed. The object data 61B, which is the object of the predetermined inference, is data related to the state of the moving body RB of the object observed by the sensor SB1. On the other hand, the environmental data 63B, which is related to the object environment in which the predetermined inference is performed, is data related to the object observation environment of the moving body RB of the object observed by the sensor SB1.

[0523] In this modified example, the prediction device 1B is further connected to other sensors SB2. These other sensors SB2 differ from the sensor SB1, which observes the state of the moving body RB; they are used to observe the object observation environment of the moving body RB, which is the object observed by sensor SB1. Therefore, the prediction device 1B can acquire environmental data 63B from the other sensors SB2. When considering weather, air pressure, etc., as the object observation environment, the other sensors SB2 can be, for example, a weather sensor, a barometer, etc.

[0524] In this variation, the prediction device 1B provides the acquired object data 61B to each learned machine learning model 45B, enabling each learned machine learning model 45B to predict the movement path of the moving body RB. Thus, the prediction device 1B obtains the prediction results of each learned machine learning model 45B regarding the movement path of the moving body RB. Then, the prediction device 1B integrates the prediction results of each learned machine learning model 45B according to the integration rule 5B.

[0525] Similar to the above-described embodiment, the integration rule 5B includes multiple integration parameters PB that specify the degree of importance given to the prediction results of each learned machine learning model 45B in the object environment. The prediction device 1B of this variant determines the values ​​of each integration parameter PB based on the environmental data 63B. At this time, in addition to the environmental data 63B, the object data 61B may also be considered. Next, the prediction device 1B of this variant uses the determined values ​​of each integration parameter PB to weight the prediction results of the corresponding learned machine learning models 45B. Then, the prediction device 1B of this variant integrates the weighted prediction results of each learned machine learning model 45B.

[0526] Therefore, the prediction device 1B involved in this variation can generate a result that predicts the movement path of the moving body RB based on the state of the moving body RB in the object environment.

[0527] <Generation of the Machine Learning Model After Learning>

[0528] In this variation, each local learning device 2B, like the local learning devices 2 described above, generates a learned machine learning model 45B by executing steps S101 to S104. Specifically, in step S101, each local learning device 2B acquires local learning data 30B. Then, in step S102, each local learning device 2B uses the acquired local learning data 30B to perform machine learning on the machine learning model 40B. The structure and method of the machine learning model 40B can be the same as those of the machine learning model 40 described above.

[0529] In this variation, the machine learning method can be any of the first to third examples described above. In the case of the first example, the local learning data 30B consists of multiple learning datasets, each including a combination of training data and forward tracing data. The training data is the same as the object data 61B described above and can consist of data related to the state of the moving body RB observed by sensor SB1. The forward tracing data can consist of data representing the inference result (forward tracing) from the training data, i.e., the path along which the moving body RB actually or virtually moves in that state. Each learning dataset can be generated using the same method as in the embodiments described above. Each local learning device 2B performs supervised learning of the machine learning model 40B using the acquired local learning data 30B. The supervised learning method can be the same as in the embodiments described above.

[0530] In the second example, the local learning data 30B consists of multiple training data sets. The training data consists of data that the machine learning model 40B is intended to generate. For example, the training data consists of data representing the actual or virtual movement path of the moving body RB. Each training data set can be generated using the same method as in the above embodiment. Each local learning device 2B uses the acquired local learning data 30B to perform adversarial learning against the machine learning model 40B and other machine learning models. The adversarial learning method can be the same as in the above embodiment.

[0531] In the third example, the machine learning model 40B can be value-based, policy-based, or both. The state that becomes the object of observation can be a state related to the movement path of the mobile body RB, and the action performed by the agent can be the movement of the mobile body RB. The reward function can also be manually set by an operator, etc. Alternatively, the reward function can be set to provide an immediate reward based on the appropriateness of the prediction result of the machine learning model 40B. In this case, similarly to the above embodiment, the appropriateness can also be manually provided by an operator, etc. Alternatively, the appropriateness can also be evaluated using a decision maker according to a prescribed benchmark. The decision maker can be configured to evaluate the accuracy of the result of predicting the movement path of the mobile body RB. Correspondingly, the reward function can be set to provide a positive immediate reward if the accuracy of the path prediction is above a threshold, and a negative immediate reward if the accuracy of the path prediction is below an acceptable value, based on the trial results of multiple prediction processes. Alternatively, the reward function can also be inferred from case data representing the demonstration of an expert through inverse reinforcement learning. In this variant example, the case data can, for example, consist of data representing the actual movement path of the mobile body RB. Each local learning device 2B adjusts the values ​​of the operational parameters of the machine learning model 40B to maximize the value (expected value) obtained in an appropriately set learning environment. The reinforcement learning method can be the same as the above-described implementation.

[0532] Each local learning device 2B can perform machine learning on the machine learning model 40B using any of the methods described above. Thus, each local learning device 2B can generate a learned machine learning model 45B that has acquired the ability to predict the movement path of the moving body RB based on its state.

[0533] In step S103, each local learning device 2B acquires learning environment data 35B related to the environment in which the local learning data 30B is obtained. The learning environment data 35B is the same type of data as the environment data 63B obtained by the prediction device 1B. The method for acquiring the learning environment data 35B can be the same as the method for acquiring the environment data 63B. For example, each local learning device 2B can be further connected to a sensor of the same type as other sensors SB2, and can also acquire the learning environment data 35B from that sensor. In step S104, each local learning device 2B generates information related to the generated, learned machine learning model 45B as learning result data 47B. Then, each local learning device 2B stores the learning result data 47B and the learning environment data 35B in a designated storage area in an associated manner.

[0534] In this variation, local learning data 30B can be acquired in different environments among the various local learning devices 2B. Then, a learned machine learning model 45B can be generated based on the acquired local learning data 30B. As a result, multiple learned machine learning models 45B derived from the local learning data 30B acquired in different environments are obtained, capable of predicting the movement path of the moving body RB.

[0535] <Hardware Configuration of the Prediction Device>

[0536] Figure 18 An example of the hardware configuration of the prediction device 1B involved in this variation is illustrated schematically. For example... Figure 18 As shown, the prediction device 1B involved in this variation, like the inference device 1 described above, is a computer electrically connected to the control unit 11, storage unit 12, communication interface 13, external interface 14, input device 15, output device 16, and driver 17. The prediction device 1B is connected to sensor SB1 and other sensors SB2 via the external interface 14. However, the hardware configuration of the prediction device 1B is not limited to this example. Regarding the specific hardware configuration of the prediction device 1B, constituent elements can be appropriately omitted, substituted, or added according to the implementation method. Besides being an information processing device designed specifically for the services provided, the prediction device 1B can be a general-purpose server device, a general-purpose PC, etc.

[0537] In this modified example, the storage unit 12 of the prediction device 1B stores various information such as prediction program 81B, integrated rule data 57B, learning result data 47B, learning data 59B, and learning environment data 35B. The prediction program 81B, integrated rule data 57B, learning result data 47B, learning data 59B, and learning environment data 35B correspond to the inference program 81, integrated rule data 57, learning result data 47, learning data 59, and learning environment data 35B described in the above-described embodiment. At least any one of the prediction program 81B, integrated rule data 57B, learning result data 47B, learning data 59B, and learning environment data 35B can also be stored in the storage medium 91. Furthermore, the prediction device 1B can also obtain at least any one of the prediction program 81B, integrated rule data 57B, learning result data 47B, learning data 59B, and learning environment data 35B from the storage medium 91.

[0538] <Software Structure of the Prediction Device>

[0539] Figure 19A and Figure 19B An example of the software configuration of the prediction device 1B according to this modification is illustrated schematically. Similar to the embodiment described above, the software configuration of the prediction device 1B is implemented by the control unit 11 executing the prediction program 81B. For example... Figure 19A and Figure 19B As shown, except for the limitation on the data processed and the content of the inference, the software configuration of the prediction device 1B is the same as that of the inference device 1. Therefore, the prediction device 1B performs the same series of processes related to the inference as the inference device 1.

[0540] (Step S201)

[0541] That is, such as Figure 19A As shown, in step S201, the control unit 11 of the prediction device 1B acquires object data 61B. Object data 61B can be any data related to the state of the moving body RB observed by the sensor SB1, and its content is not particularly limited and can be appropriately selected according to the implementation method. Object data 61B may include, for example, observation data obtained by the sensor SB1 that observes the state of the moving body RB, data indicating the state of the movement range analyzed based on the observation data (e.g., congestion), data indicating the current position of the moving body RB, and data indicating the movement path of the moving body RB up to the present, etc.

[0542] The method for acquiring object data 61B is not particularly limited and can be appropriately selected depending on the implementation method. For example, observation data from sensor SB1 can be acquired as object data 61B. Alternatively, object data 61B can be acquired by performing some analysis processing on the observation data obtained from sensor SB1. Furthermore, object data 61B can be acquired through input from an operator or the like. The control unit 11 can acquire object data 61B directly from each device or indirectly via another computer.

[0543] (Step S202)

[0544] In step S202, the control unit 11 acquires environmental data 63B. Environmental data 63B can be any data related to the observation environment of the moving body RB, the object observed by sensor SB1, and its content is not particularly limited and can be appropriately selected according to the implementation method. Environmental data 63B may include, for example, data representing the attributes of the moving body RB, data representing the attributes of the movement range, data representing environmental attributes affecting the movement, data representing attributes related to the specifications (or performance) of sensor SB1, and data representing attributes related to the observation conditions of sensor SB1. The attributes of the moving body RB may include, for example, the type of the moving body RB (distinction between a person and a vehicle, vehicle type, etc.) and information related to the movement capability of the moving body RB. If the moving body RB is a person, the movement capability can also be evaluated based on the person's gender, age, height, weight, etc. Furthermore, if the moving body RB is a vehicle, the movement capability can be evaluated based on the vehicle's performance. The attributes of the movement range may include, for example, the location of the movement and the type of the movement route (road). The type of movement route (road) may include, for example, pedestrian-only roads, highways, and ordinary roads. Environmental attributes affecting movement may include, for example, weather, congestion, day of the week, and holiday / weekday distinctions. Attributes related to the specifications of sensor SB1 may include, for example, the sensor SB1's sensitivity limits, dynamic range, settable range of spatial resolution, and settable range of sampling frequency. Attributes related to the observation conditions of sensor SB1 may include, for example, the sensor SB1's setup angle and attributes related to sensor SB1's motion settings. Attributes related to sensor SB1's motion settings may include, for example, the sensor SB1's measurement range settings, measurement range resolution settings, and sampling frequency settings. As a specific example, if sensor SB1 is a camera, motion setting-related attributes may include aperture value, shutter speed, and zoom ratio.

[0545] The method for acquiring environmental data 63B is not particularly limited and can be appropriately selected depending on the implementation method. For example, environmental data 63B can be acquired through input from an operator or the like. Alternatively, environmental data 63B can also be acquired by performing some analysis processing on the observation data obtained by sensor SB1. Alternatively, environmental data 63B can also be acquired from other sensors (e.g., other sensor SB2) that observe the environment from the object being observed. Alternatively, environmental data 63B can also be acquired from other information processing devices (servers) that publish information such as meteorological data. The control unit 11 can acquire environmental data 63B directly from each device or indirectly via other computers.

[0546] (Step S203)

[0547] In step S203, the control unit 11 sets up each learned machine learning model 45B by referring to each set of learning result data 47B. Next, the control unit 11 provides object data 61B to each learned machine learning model 45B and performs computational processing on each learned machine learning model 45B. Thus, the control unit 11 obtains the prediction results of each learned machine learning model 45B for the movement path of the moving body RB, and uses this as the output of each learned machine learning model 45B.

[0548] (Step S204)

[0549] In step S204, the control unit 11 sets the integration rule 5B with reference to the integration rule data 57B. Then, the control unit 11 integrates the prediction results of each learned machine learning model 45B according to the integration rule 5B. Specifically, in step S211, the control unit 11 determines the value of each integration parameter PB based on the environmental data 63B. At this time, in addition to the environmental data 63B, the object data 61B may also be considered. The method for determining the value of each integration parameter PB can be any of the first to third methods described below.

[0550] (1) First method

[0551] like Figure 19B As shown, when using the first method, the control unit 11 acquires learning data 59B. Then, the control unit 11 uses the learning data 59B to perform machine learning on the computational model 51B. The configuration of the computational model 51B and the machine learning method can be the same as those of the computational model 51 described above. The configuration of the computational model 51B and the machine learning method can adopt either of the two examples described above.

[0552] In the first example, learning data 59B, like learning data 59 described above, can be composed of multiple learning datasets, each including combinations of training environment data, training object data, and forward tracing data. The training environment data is the same type of data as environment data 63B. The training object data is the same type of data as object data 61B. The forward tracing data can be composed of data representing the inference result (forward tracing) from the training object data, i.e., the actual or virtual path of the moving body RB in any state within the object environment. Each learning dataset can be generated using the same method as in the above embodiment. The control unit 11 uses the acquired learning data 59B to perform machine learning on the computational model 51B. The machine learning method can be the same as in the first example described above.

[0553] In the second example, the computational model 51B can be value-based, policy-based, or both. Similar to the above implementation, the state of the observed object corresponds to the environmental data 63B and the object data 61B. The action performed by the agent can be the movement of the mobile body RB. The agent obtains the prediction results of each learned machine learning model 45B through the above series of processes, and by integrating the obtained prediction results, it can generate a prediction result of the movement path of the mobile body RB in the object environment. The agent can also determine the action to be taken based on the generated prediction result.

[0554] The reward function can also be manually set by the operator. Alternatively, the reward function can be set to provide an immediate reward based on the appropriateness of the prediction results generated through the above series of processes. The appropriateness can also be manually provided by the operator. Alternatively, the appropriateness can be evaluated using a decision maker according to a prescribed benchmark. The decision maker can be the same as the decision maker in the reinforcement learning of the machine learning model 40B described above. Alternatively, the reward function can also be inferred from example data representing expert demonstrations through inverse reinforcement learning. This example data can be the same as the example data used in setting the reward function in the reinforcement learning of the machine learning model 40B described above. The control unit 11 adjusts the values ​​of the operation parameters of the operation model 51B to maximize the value (expected value) obtained in an appropriately set learning environment. The reinforcement learning method can be the same as the above-described implementation method.

[0555] The control unit 11 can perform machine learning on the computational model 51B using any of the methods described above. Thus, the control unit 11 can generate a learned computational model 52B that has acquired the ability to infer the values ​​of each integrated parameter PB based on environmental data 63B (and object data 61B) to predict the movement path of the moving body RB suitable for the object observation environment. The control unit 11 can also store information related to the generated learned computational model 52B in a designated storage area. The information related to the learned computational model 52B can be stored as at least a part of the integrated rule data 57B, or it can be stored separately from the integrated rule data 57B.

[0556] In the first method, the control unit 11 determines the values ​​of each integration parameter PB using the learned computational model 52B generated above. That is, in step S211, the control unit 11 provides environmental data 63B to the learned computational model 52B. At this time, the control unit 11 can further provide object data 61B to the learned computational model 52B. Then, the control unit 11 performs computational processing on the learned computational model 52B. As a result, the control unit 11 can obtain the values ​​of each integration parameter PB for each learned machine learning model 45B, as the output of the learned computational model 52B.

[0557] (2) Second method

[0558] In the second method, the control unit 11 determines the value of each integration parameter PB based on a comparison between the object environment and each learning environment. Specifically, the control unit 11 acquires the learning environment data 35B of each learned machine learning model 45B. Next, the control unit 11 calculates the fitness of each set of learning environment data 35B and environment data 63B. The form and representation of the fitness can be the same as in the above embodiment. In step S211 above, the control unit 11 can also determine the value of each integration parameter PB for each corresponding learned machine learning model 45B based on the calculated fitness. The method for determining the value of the integration parameter PB based on the fitness can be the same as in the above embodiment.

[0559] (3) Third method

[0560] In the third method, the control unit 11 receives the operator's specification of the value of the integration parameter PB, and sets the value of the object's integration parameter PB to the specified value. The receiving screen can be the same as in the above embodiment. Figure 12A ).

[0561] Return to Figure 19AIn step S211, the control unit 11 determines the value of each integration parameter PB by employing at least one of the three methods described above. In step S212, the control unit 11 uses the determined values ​​of each integration parameter PB to weight the prediction results of the corresponding learned machine learning models 45B. If at least one integration parameter PB is set using the third method described above, in step S212, the control unit 11 uses the set value to weight the prediction results of the learned machine learning models 45B corresponding to the integration parameter PB of the object. In step S213, the control unit 11 integrates the weighted prediction results of each learned machine learning model 45B. The integration of prediction results can be performed using a weighted average or a weighted majority vote, similar to the above embodiment. Therefore, it is possible to generate a result predicting the movement path of the moving body RB based on its state in the object environment.

[0562] (Step S205)

[0563] In step S205, the control unit 11 outputs information related to the generated prediction result. Similar to the above embodiment, the output destination and the content of the output information can be appropriately determined according to the embodiment. The control unit 11 may also directly output the prediction result generated in step S204 to the output device 16, or it may perform certain information processing based on the generated prediction result.

[0564] For example, the control unit 11 may also generate recommended movement paths, such as those indicating less congested routes, based on the prediction results generated in step S204 as information related to the prediction results. Then, the control unit 11 may also output the generated information indicating recommended paths to other moving bodies.

[0565] It should be noted that, similar to the embodiments described above, the control unit 11 of the prediction device 1B can assign each learned machine learning model 45B to at least one of the multiple groups. Therefore, the control unit 11 can also perform integration processing for each group according to the purpose. The grouping method can be any of the two methods described above.

[0566] <Inferential processing of local learning devices>

[0567] In addition, the local learning devices 2B involved in this variation are similar to the local learning devices 2 described above. By executing the processing steps S111 to S113, they can use the learned machine learning model 45B to predict the movement path of the moving body RB.

[0568] In step S111, each local learning device 2B acquires the object data used in the prediction. The method for acquiring the object data can be the same as step S201 of the prediction device 1B described above. In step S112, each local learning device 2B provides the object data to the learned machine learning model 45B and performs the computational processing of the learned machine learning model 45B. As a result, each local learning device 2B acquires the result of predicting the moveme...

Claims

1. A reasoning device, comprising: The first data acquisition department acquires the specified object data that becomes the object of the specified inference; The second data acquisition unit acquires environmental data related to the object environment in which the prescribed inference is performed; The result acquisition unit provides the object data to multiple inference models derived from local learning data obtained in different environments to perform the prescribed inference, and causes each inference model to perform the prescribed inference on the object data, thereby acquiring the inference result of each inference model on the acquired object data. The integration department generates inference results for the object data within the object environment by integrating the inference results of each inference model according to integration rules. The integration rule consists of multiple integration parameters that specify the degree of importance attached to the inference results of each inference model in the context of the object. The inference results of integrating the inference models according to the aforementioned integration rules have the following: The values ​​of each of the integrated parameters are determined based on the acquired environmental data. The inference results of each of the inference models are weighted using the determined values ​​of each of the aforementioned integration parameters, and The inference results are obtained by integrating the weighted sum of the various inference models. as well as The output section outputs information related to the generated inference results. Each of the aforementioned inference models consists of a fully learned machine learning model that has acquired the ability to perform the specified inferences through machine learning using the local learning data. The values ​​of each of the integration parameters are determined in a manner that emphasizes the inference results of the learned machine learning model in a learning environment similar to the described object environment. The values ​​of each integration parameter are determined based on the environmental data by acquiring learning environment data related to the environment in which the local learning data used in the derivation of each inference model is obtained, calculating the fitness of the acquired learning environment data and the environmental data, and determining the values ​​of each integration parameter for each inference model based on the calculated fitness. The fitness score is used to evaluate whether the object's environment is similar to the learning environments of the multiple inference models.

2. The inference device according to claim 1, wherein, The inference device further includes a learning processing unit that performs machine learning on a computational model for calculating the values ​​of each of the integrated parameters based on the environmental data. The machine learning is configured by training the computational model so that the calculated values ​​of each of the integrated parameters are suitable for the prescribed inference in the object environment. The integration rules also include a learned computational model constructed through the machine learning process. The values ​​of each of the integrated parameters are determined based on the environmental data by providing the environmental data to the learned operational model and performing the operational processing of the learned operational model to obtain the values ​​of each of the integrated parameters as the output of the learned operational model.

3. The inference device according to claim 2, wherein, The machine learning of the computational model is constituted by rewarded reinforcement learning based on the appropriateness of the inferences generated through the integration. Training the computational model includes repeatedly adjusting the values ​​of the computational parameters constituting the computational model to obtain more of the reward.

4. The inference device according to claim 1 or 2, wherein, The inference device further includes a parameter setting unit, which receives a specified value for at least one of the plurality of integrated parameters and sets the value of the at least one integrated parameter to the specified value. Weighting the inference results of each inference model includes: using the set value to weight the inference results of the inference model corresponding to the at least one integration parameter.

5. The inference device according to claim 1 or 2, wherein, The prescribed inference is a method for inferring a change from the current observation state of the properties of the object observed by the sensor to an appropriate observation state for obtaining observation data suitable for specified conditions by the sensor. The object data is related to the current observation state of the object sensor. The environmental data is related to the object observation environment in which the object sensor observes the properties of the object.

6. The inference device according to claim 5, wherein, The environmental data is obtained from other sensors, different from the object sensor, that observe the object's observation environment.

7. The inference device according to claim 5, wherein, The inference device is connected to an intervention device that changes the observation state of the object sensor. Information related to the inference includes instructions for the intervention device to perform actions to change the observation state of the object sensor according to the inferred modification method. The output unit causes the intervention device to perform the action of changing the observation state of the object sensor according to the inferred change method by sending the instruction information to the intervention device.

8. The inference device according to claim 5, wherein, Information related to the inference results includes instructions to guide the user to change the observation state of the object sensor according to the change method. The output section outputs the instruction information to the output device.

9. The inference device according to claim 5, wherein, The sensor is a camera. The observation data is image data. The specified conditions are related to capturing the object in the image data to facilitate the examination of the object's attributes.

10. The inference device according to claim 1 or 2, wherein, The aforementioned inference is based on predicting the movement path of a moving object according to its state. The object data is related to the state of the moving object observed by the sensor. The environmental data is related to the object observation environment in which the sensor observes the movement of the moving object.

11. The inference device according to claim 1 or 2, wherein, The inference stated therein is based on the user's conversational actions, inferring a suitable conversational strategy for the user. The object data is related to the session actions of the object user. The environmental data is related to the object session environment in which the object user performs session actions.

12. The inference device according to claim 1 or 2, wherein, The aforementioned inference is based on the state of the robot device to deduce the appropriate action instructions for the robot device for the task. The object data is related to the state of the object robot device. The environmental data is related to the object task environment in which the object robot device completes the task.

13. A method of inference, wherein a computer performs the following steps: Obtain the specified object data that becomes the object of the specified inference; Acquire environmental data related to the object environment in which the prescribed inferences are performed; The object data is provided to multiple inference models derived from local learning data obtained in different environments to perform the prescribed inferences, and each inference model performs the prescribed inferences on the object data to obtain the inference results of each inference model on the obtained object data. By integrating the inference results of each inference model according to the integration rules, an inference result for the object data in the object environment is generated. The integration rule consists of multiple integration parameters that specify the degree of importance attached to the inference results of each inference model in the context of the object. The inference results of integrating the inference models according to the aforementioned integration rules have the following: The values ​​of each of the integrated parameters are determined based on the acquired environmental data. The inference results of each of the inference models are weighted using the determined values ​​of each of the aforementioned integration parameters, and The inference results are obtained by integrating the weighted sum of the various inference models. as well as Output information related to the generated inference results. Each of the aforementioned inference models consists of a fully learned machine learning model that has acquired the ability to perform the specified inferences through machine learning using the local learning data. The values ​​of each of the integration parameters are determined in a manner that emphasizes the inference results of the learned machine learning model in a learning environment similar to the described object environment. The values ​​of each integration parameter are determined based on the environmental data by acquiring learning environment data related to the environment in which the local learning data used in the derivation of each inference model is obtained, calculating the fitness of the acquired learning environment data and the environmental data, and determining the values ​​of each integration parameter for each inference model based on the calculated fitness. The fitness score is used to evaluate whether the object's environment is similar to the learning environments of the multiple inference models.

14. A program product comprising a deduction program for causing a computer to perform the following steps: Obtain the specified object data that becomes the object of the specified inference; Acquire environmental data related to the object environment in which the prescribed inferences are performed; The object data is provided to multiple inference models derived from local learning data obtained in different environments to perform the prescribed inferences, and each inference model performs the prescribed inferences on the object data to obtain the inference results of each inference model on the obtained object data. By integrating the inference results of each inference model according to the integration rules, an inference result for the object data in the object environment is generated. The integration rule consists of multiple integration parameters that specify the degree of importance attached to the inference results of each inference model in the context of the object. The inference results of integrating the inference models according to the aforementioned integration rules have the following: The values ​​of each of the integrated parameters are determined based on the acquired environmental data. The inference results of each of the inference models are weighted using the determined values ​​of each of the aforementioned integration parameters, and The inference results are obtained by integrating the weighted sum of the various inference models. as well as Output information related to the generated inference results. Each of the aforementioned inference models consists of a fully learned machine learning model that has acquired the ability to perform the specified inferences through machine learning using the local learning data. The values ​​of each of the integration parameters are determined in a manner that emphasizes the inference results of the learned machine learning model in a learning environment similar to the described object environment. The values ​​of each integration parameter are determined based on the environmental data by acquiring learning environment data related to the environment in which the local learning data used in the derivation of each inference model is obtained, calculating the fitness of the acquired learning environment data and the environmental data, and determining the values ​​of each integration parameter for each inference model based on the calculated fitness. The fitness score is used to evaluate whether the object's environment is similar to the learning environments of the multiple inference models.

Images