Information processing method and information processing system

The method improves inference model selection by using a first inference result to choose a suitable second model, addressing inefficiencies in existing technologies and reducing computational costs in resource-constrained devices.

JP7576765B2Active Publication Date: 2024-11-01PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2022534938
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-07-10
Filing Date
2021-05-24
Publication Date
2024-11-01
Estimated Expiration
2041-05-24

AI Technical Summary

Technical Problem

Existing inference processing technologies often select inappropriate models due to indirect data selection, leading to high computational costs and inefficiencies in handling various recognition targets and difficulty levels, especially in resource-constrained devices.

Method used

An information processing method that selects an appropriate inference model by inputting sensing data into a first model, obtaining a first inference result, and using this result to select a suitable second model based on model selection information, thereby reducing unnecessary computational resources and improving model suitability.

Benefits of technology

This method enhances the likelihood of selecting an appropriate inference model for the data, reducing calculation costs while enabling efficient processing of diverse targets, and improving training efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007576765000001
    Figure 0007576765000001
  • Figure 0007576765000002
    Figure 0007576765000002
  • Figure 0007576765000003
    Figure 0007576765000003
Patent Text Reader

Abstract

In the present invention, a feature map that is a first inference result with respect to sensing data is acquired from a first inference model by inputting the sensing data to the first inference model (S2002), model selection information indicating at least one second inference model selected from among a plurality of the second inference models is acquired from a third inference model by inputting the first inference result to the third inference model (S2003), and by inputting the first inference result to at least one of the second inference models indicated by the model selection information, at least one second inference result with respect to the first inference result is acquired from the second inference model and is outputted (S2005).
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing method and an information processing system. [Background technology]

[0002] There is a technology that executes inference processing based on data acquired from a device (see Patent Document 1). This document discloses that an inference model that executes inference processing is selected based on the environment or conditions in which the device is placed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 173121 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology of the above document, an inference model is selected based on indirect information about the data obtained from the device, so that an inference model suitable for the data to be subjected to inference processing (in other words, the input data for the inference model) may not be selected.

[0005] Therefore, the present invention provides an information processing method that can improve the likelihood of selecting an inference model suitable for the data to be subjected to inference processing. [Means for solving the problem]

[0006] An information processing method according to one embodiment of the present invention is an information processing method executed by a processor using a memory, which inputs sensing data into a first inference model, thereby obtaining a first inference result for the sensing data from the first inference model, inputs the first inference result into a third inference model, thereby obtaining model selection information from the third inference model indicating at least one second inference model selected from a plurality of second inference models, inputs the first inference result into the at least one second inference model indicated by the model selection information, thereby obtaining at least one second inference result for the first inference result from the second inference model, and outputs the at least one second inference result.

[0007] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0008] The information processing method of the present invention can improve the likelihood of selecting an inference model suitable for the data to be subjected to inference processing. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a functional configuration of the learning system according to the first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram showing an example of an image input to the learning system in the first embodiment. [Diagram 3] FIG. 3 is an explanatory diagram of an example of task evaluation information according to the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram illustrating an example of training data in the first embodiment. [Diagram 5] FIG. 5 is a flow diagram showing the processing executed by the learning system in the first embodiment. [Figure 6] FIG. 6 is a block diagram showing a functional configuration of the inference system in the first embodiment. [Figure 7] FIG. 7 is a flow diagram showing the processing executed by the inference system in the first embodiment. [Figure 8] FIG. 8 is a block diagram showing a functional configuration of the training system in the second embodiment. [Figure 9] FIG. 9 is an explanatory diagram illustrating an example of training data in the second embodiment. [Figure 10] FIG. 10 is a flow diagram showing the processing executed by the training system in the second embodiment. [Figure 11] FIG. 11 is a block diagram showing a functional configuration of the inference system according to the second embodiment. [Figure 12] FIG. 12 is a flow diagram showing the processing executed by the inference system in the second embodiment. [Figure 13] FIG. 13 is a block diagram showing a functional configuration of the training system in the third embodiment. [Figure 14] FIG. 14 is a flow diagram showing the processing executed by the training system in the third embodiment. [Figure 15] FIG. 15 is a block diagram showing the functional configuration of the inference system according to the third embodiment. [Figure 16] FIG. 16 is a flow diagram showing the processing executed by the inference system in the third embodiment. [Figure 17] FIG. 17 is a block diagram showing a functional configuration of the training system in the fourth embodiment. [Figure 18] FIG. 18 is a flow diagram showing the processing executed by the training system in the fourth embodiment. [Figure 19] FIG. 19 is a block diagram showing the functional configuration of the inference system according to the fourth embodiment. [Figure 20] FIG. 20 is a flow diagram showing the processing executed by the inference system in the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (Findings on which the present invention is based) In recent years, with the improvement of inference processing technology using machine learning and the like, cases in which inference processing is applied are expanding. For example, inference processing is used for recognizing human behavior in the home. On the other hand, in behavior recognition, there are objects or scenes of various types or difficulty levels as recognition targets. In addition, in order to recognize behavior in the home, it is required to reduce the size and cost of the device that performs the recognition processing, and the computational resources of the device are limited. Therefore, in behavior recognition in the home, it is required to recognize various recognition targets with limited computational resources (e.g., processor, memory). It is assumed that this is not limited to behavior recognition, but also applies to other inference processing.

[0011] However, in order to handle various types of recognition targets, many types of behavior recognition models must be used to perform the recognition process, which results in huge computational costs. Also, in order to handle recognition targets with various levels of difficulty, a huge behavior recognition model must be used to perform the recognition process, which results in huge computational costs (e.g., computational amount, computation time, computational resource occupation, power consumption).

[0012] Furthermore, a conventional technique involves selecting an inference model for executing inference processing based on the environment or conditions in which the device is placed.

[0013] However, in the above technology, since an inference model is selected based on indirect information for data acquired from a device, an inference model suitable for the data to be inferred may not be selected. In addition, since the inference model selection process and the inference process are performed independently, calculation costs are high.

[0014] Therefore, an information processing method according to one embodiment of the present invention is an information processing method executed by a processor using a memory, which inputs sensing data into a first inference model, thereby obtaining a first inference result for the sensing data from the first inference model, inputs the first inference result into a third inference model, thereby obtaining model selection information from the third inference model indicating at least one second inference model selected from a plurality of second inference models, inputs the first inference result into the at least one second inference model indicated by the model selection information, thereby obtaining at least one second inference result for the first inference result from the second inference model, and outputs the at least one second inference result.

[0015] According to the above aspect, since the second inference result is obtained using at least one second inference model selected from the plurality of second inference models, when the second inference result is obtained, the second inference model that is not selected from the plurality of second inference models is not operated. Therefore, the computational resources or power required for the operation of the second inference model that is not selected can be reduced. In this way, the above information processing method can suppress the computational cost of the inference process while enabling inference processing for various inference targets. In addition, since the second inference model is selected based on the first inference result obtained by inputting the sensing data, the second inference model is substantially selected based on the characteristics of the sensing data. Therefore, it is possible to improve the possibility that an inference model suitable for the data to be inferred can be selected.

[0016] For example, the third inference model may be a model trained using machine learning based on at least one second training inference result obtained by inputting training sensing data into the first inference model, obtaining a first training inference result for the training sensing data from the first inference model, and inputting the first training inference result into at least one selected second inference model for each selection pattern of at least one second inference model among the multiple second inference models, and training model selection information, which is the model selection information obtained by inputting the first training inference result into the third inference model.

[0017] According to the above aspect, by using machine learning, it is possible to generate a third inference model that performs the complex inference process of selecting a second inference model from the characteristics of sensing data more easily than with other techniques.

[0018] For example, training of the third inference model may include training using machine learning in which data generated in accordance with the format of the model selection information based on the at least one training second inference result is used as reference data, and the training model selection information is used as output data.

[0019] According to the above aspect, by generating data conforming to the format of the model selection information from the second inference result for training, it is possible to train the third inference model without being restricted by the format of the second inference result for training. In other words, various second inference models that output various second inference results can be applied to the embodiments of the present disclosure.

[0020] For example, the model selection information may include first information corresponding to a task to be performed by a second inference model, and the at least one second inference model indicated by the model selection information may perform a task corresponding to the first information, and the reference data may be generated so that a second inference model performing a task that contributed to the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information.

[0021] According to the above aspect, the second inference model that executes a task that contributed to at least one second inference result is preferentially included in training model selection information as reference data for machine learning, so that it is possible to automatically generate a third inference model that outputs model selection information for selecting the second inference model that executes the contributing task. In addition, the reference data contains more data that is effective for training the third inference model, improving the efficiency of training.

[0022] For example, generating the reference data may include generating the reference data such that second inference models that perform tasks belonging to a task set having a score based on the at least one training second inference result higher than a threshold are preferentially included in the at least one second inference model indicated by the training model selection information.

[0023] According to the above aspect, the second inference model that executes a task belonging to a task set whose score is higher than a threshold is preferentially included in training model selection information as reference data during machine learning, so that the level of precision or accuracy of the second inference result by the second inference model that is actually selected by the third inference model can be brought closer to a predetermined standard.

[0024] For example, the model selection information may include second information corresponding to performance of the second inference model, at least one second inference model indicated by the model selection information has performance corresponding to the second information, and generating the reference data may include generating the reference data such that a second inference model that satisfies a performance requirement in the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information.

[0025] According to the above aspect, the second inference model that satisfies the performance requirements is preferentially included in the training model selection information as reference data for machine learning, so that it is possible to automatically generate a third inference model that outputs model selection information that selects the second inference model that satisfies the performance requirements. In addition, the reference data contains more data that is effective for training the third inference model, improving the efficiency of training.

[0026] For example, the second information may include a difficulty of processing the second inference model, and generating the reference data may include generating the reference data so that a second inference model having performance corresponding to the difficulty is preferentially included in at least one second inference model indicated by the training model selection information.

[0027] According to the above aspect, the second inference model having performance according to the level of difficulty is preferentially included in the training model selection information as reference data during machine learning, so that the processing performance (in other words, the computational cost) of the inference processing by the second inference model that is actually selected by the third inference model can be brought closer to the required level.

[0028] For example, the second inference model may include a neural network model, the second information may include size information indicating the size of the neural network model as the performance of the second inference model, and generating the reference data may include generating the reference data such that a second inference model having the size information included in the second information is preferentially included in at least one second inference model indicated by the training model selection information.

[0029] According to the above aspect, since the size information of the neural network is used as the performance of the second inference model, the performance of the second inference model can be adjusted more easily.

[0030] Furthermore, an information processing method according to one embodiment of the present invention is an information processing method executed by a processor using a memory, which inputs sensing data into a first inference model to obtain from the first inference model a first inference result for the sensing data and model selection information indicating at least one second inference model selected from a plurality of second inference models, inputs the first inference result to the at least one second inference model indicated by the model selection information to obtain at least one second inference result for the first inference result from the second inference model, and outputs the at least one second inference result.

[0031] According to the above aspect, the second inference result is obtained using at least one second inference model selected from a plurality of second inference models, so that when the second inference result is obtained, a second inference model that is not selected from the plurality of second inference models is not operated. Therefore, it is possible to reduce the computational resources or power required for the operation of the second inference model that is not selected. In this way, the above information processing method can suppress the computational costs involved in the inference process while enabling inference processing for various inference targets. In addition, since the second inference model is selected substantially based on the sensing data, it is possible to improve the possibility that an inference model suitable for the data of the inference processing target is selected.

[0032] For example, the first inference model may be a model trained using machine learning based on at least one second training inference result obtained by inputting training sensing data into the first inference model, obtaining a first training inference result for the training sensing data and training model selection information from the first inference model, and inputting the first training inference result into at least one selected second inference model for each selection pattern of at least one second inference model among the multiple second inference models, and the training model selection information.

[0033] According to the above aspect, by using machine learning, an inference model that performs complex inference processing of selecting a second inference model from the characteristics of sensing data can be generated as a first inference model more easily than with other techniques.

[0034] For example, training of the first inference model may include training using machine learning in which data generated in accordance with the format of the model selection information based on the at least one training second inference result is used as reference data, and the training model selection information is used as output data.

[0035] According to the above aspect, by generating data conforming to the format of the model selection information from the second inference result for training, it is possible to train the first inference model without being restricted by the format of the second inference result for training. In other words, various second inference models that output various second inference results can be applied to the embodiments of the present disclosure.

[0036] For example, the model selection information may include first information corresponding to a task to be performed by a second inference model, and the at least one second inference model indicated by the model selection information may perform a task corresponding to the first information, and the reference data may be generated so that a second inference model performing a task that contributed to the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information.

[0037] According to the above aspect, the second inference model that executes a task that contributed to at least one second inference result is preferentially included in training model selection information as reference data for machine learning, so that it is possible to automatically generate a first inference model that outputs model selection information for selecting the second inference model that executes the contributing task. In addition, the reference data contains more data that is effective for training the first inference model, improving the efficiency of training.

[0038] For example, generating the reference data may include generating the reference data such that second inference models that perform tasks belonging to a task set having a score based on the at least one training second inference result higher than a threshold are preferentially included in the at least one second inference model indicated by the training model selection information.

[0039] According to the above aspect, a second inference model that executes a task belonging to a task set whose score is higher than a threshold is preferentially included in training model selection information as reference data during machine learning, so that the level of precision or accuracy of the second inference result by the second inference model that is actually selected by the first inference model can be brought closer to a predetermined standard.

[0040] For example, the model selection information may include second information corresponding to performance of the second inference model, at least one second inference model indicated by the model selection information has performance corresponding to the second information, and generating the reference data may include generating the reference data such that a second inference model that satisfies a performance requirement in the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information.

[0041] According to the above aspect, the second inference model that satisfies the performance requirements is preferentially included in the training model selection information as reference data for machine learning, so that it is possible to automatically generate a first inference model that outputs model selection information that selects the second inference model that satisfies the performance requirements. In addition, the reference data contains more data that is effective for training the first inference model, improving the efficiency of training.

[0042] For example, the second information may include a difficulty of processing the second inference model, and generating the reference data may include generating the reference data so that a second inference model having performance corresponding to the difficulty is preferentially included in at least one second inference model indicated by the training model selection information.

[0043] According to the above aspect, the second inference model having performance according to the difficulty level is preferentially included in the training model selection information as reference data during machine learning, so that the processing performance (in other words, the computational cost) of the inference processing by the second inference model that is actually selected by the first inference model can be brought closer to the required level.

[0044] For example, the second inference model may include a neural network model, the second information may include size information indicating the size of the neural network model as the performance of the second inference model, and generating the reference data may include generating the reference data such that a second inference model having the size information included in the second information is preferentially included in at least one second inference model indicated by the training model selection information.

[0045] According to the above aspect, since the size information of the neural network is used as the performance of the second inference model, the performance of the second inference model can be adjusted more easily.

[0046] In addition, an information processing device according to one embodiment of the present invention is an information processing system including a first acquisition unit that acquires a first inference result for sensing data from a first inference model by inputting sensing data into the first inference model, a second acquisition unit that acquires model selection information indicating at least one second inference model selected from a plurality of second inference models by inputting the first inference result into a third inference model, a third acquisition unit that acquires at least one second inference result for the first inference result from the second inference model by inputting the first inference result to the at least one second inference model indicated by the model selection information, and an output unit that outputs the at least one second inference result.

[0047] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0048] In addition, an information processing device according to one embodiment of the present invention is an information processing system including a first acquisition unit that acquires a first inference result for the sensing data and model selection information indicating at least one second inference model selected from a plurality of second inference models from a first inference model by inputting sensing data into the first inference model, a second acquisition unit that acquires at least one second inference result for the first inference result from the second inference model by inputting the first inference result to the at least one second inference model indicated by the model selection information, and an output unit that outputs the at least one second inference result.

[0049] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0050] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, or a recording medium.

[0051] Hereinafter, the embodiment will be specifically described with reference to the drawings.

[0052] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component arrangement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing a top concept are described as optional components.

[0053] (Embodiment 1) In this embodiment, an information processing method and an information processing system for efficiently operating an inference model will be described. Hereinafter, the information processing method will be simply referred to as a processing method, and the information processing system will be simply referred to as a processing system.

[0054] In this embodiment, a training system 1 and an inference system 2 will be described. The training system 1 is a processing system that trains a selection model that selects an inference model to be used for inference processing on input sensing data by machine learning. The inference system 2 is a processing system that executes inference processing on the sensing data by using an inference model selected by using a task selection model learned by the training system 1 according to the input sensing data. Hereinafter, the training system 1 and the inference system 2 will be described in order.

[0055] (1-1) Training System 1 FIG. 1 is a block diagram showing the configuration of a training system 1 which is a processing system in this embodiment.

[0056] As shown in FIG. 1, the training system 1 outputs an inference model (task selection model M1) that selects an inference model that executes inference processing on the input sensing data 10, based on the input sensing data 10.

[0057] The training system 1 includes a feature extraction unit 101, an inference control unit 102, an integration unit 103, a task selection answer generation unit 104, a task selection model training unit 105, and a model library 30. Each functional unit included in the training system 1 can be realized by a processor (e.g., a CPU (Central Processing Unit)) (not shown) executing a predetermined program using a memory. The training system 1 may be realized as one device, or may be realized by multiple devices capable of communicating with each other.

[0058] The feature extraction unit 101 is a functional unit that extracts features (first inference result) of the sensing data 10 using a feature extraction model L1 (first inference model). Specifically, the feature extraction unit 101 extracts features of the sensing data 10 by inputting the input sensing data 10 to the feature extraction model L1. For example, the feature may be a feature map. The generated feature map is provided to the inference control unit 102. The feature extraction model is, for example, a neural network model trained to output a feature map of the input data.

[0059] The sensing data 10 is, for example, image data. In the following, the sensing data 10 is described as an example of image data, but is not limited thereto. In addition to the above, the sensing data 10 may be any sensing data 10 from which correct answer data can be obtained, such as voice data output from a microphone, point cloud data output from a radar such as LiDAR (Light Detection and Ranging), pressure data output from a pressure sensor, temperature data or humidity data output from a temperature sensor or humidity sensor, or scent data output from a scent sensor.

[0060] The inference control unit 102 is a functional unit that controls inference by an inference process that uses a feature map that uses one or more task-specific inference models (second inference models) as input. The task-specific inference model is a model that infers information about an object or person contained in an image that is the sensing data 10 related to the input feature map based on the input feature map, and is, for example, a neural network model.

[0061] The inference control unit 102 acquires a feature map from the feature extraction unit 101. The inference control unit 102 also executes an inference process using a plurality of task inference models held in the model library 30. The plurality of task inference models include task inference models for executing various tasks. Here, the plurality of tasks will be described as including, but not limited to, task A of object detection, task B of pose estimation, and task C of attribute determination.

[0062] For example, task A of object detection is a task of detecting an object (e.g., a lighting device, a shelf, a drawer, a bag, etc., which may include a person) shown in an image that is the input sensing data 10. For example, task B of posture estimation is a task of estimating the posture (e.g., standing, stretching, sitting, etc.) of a person shown in the image. For example, task C of attribute determination is a task of determining the attributes of a situation shown in the image (e.g., the location is a kitchen, the time of day is morning, etc.).

[0063] The inference control unit 102 determines one or more tasks (also called a task set) selected from the multiple tasks. The inference control unit 102 also selects and determines a new task set that is at least partially different from the above task set from the multiple tasks. In this manner, the inference control unit 102 determines multiple task sets. The inference control unit 102 may determine all task sets that can be selected from the multiple tasks as the above task sets.

[0064] The inference control unit 102 executes an inference process by inputting a feature map to each of the task inference models that execute each of the tasks included in a task set selected from the determined task sets. The inference control unit 102 acquires integrated information by integrating the inference results (second inference results) output by each of the task inference models by the integration unit 103. The inference control unit 102 acquires integrated information about the determined task sets.

[0065] The integration unit 103 is a functional unit that executes an integration process based on one or more inference results and outputs integrated information. In other words, when one or more inference results are input, the integration unit 103 generates one piece of integrated information that integrates the one or more input inference results. The integration unit 103 acquires, as one or more inference results, results inferred by one or more task inference models held by the model library 30. The integrated information includes a task set, a situation (or action) estimated from the sensing data 10, and a set of scores indicating the likelihood of estimating the situation. When there are multiple situations, the integrated information includes multiple sets of situations and scores. Note that when the task set includes only one task, the integration unit 103 generates integrated information including an inference result output by a task inference model that executes the one task. The integration unit 103 can be realized by a neural network model. Specifically, the integration unit 103 may be realized by a neural network model trained to output integration information indicating a task set and a set of a situation and a score when one or more inference results are input. The inference control unit 102 provides the generated integration information to the task selection correct answer generation unit 104. Note that, here, an example will be described in which the integration process of the integration unit 103 is a process of solving a classification problem (in other words, the situation included in the integration information is the classification result), but is not limited to this. For example, the process of the integration unit 103 may be a process of solving a regression problem, an object detection problem, etc., which will be described later. Also, when the integration unit 103 processes only a single situation, the integration information does not need to include the situation. Note that the integration process in the integration unit 103 may be performed by a machine learning model.

[0066] Here, the generation of the integrated information will be specifically described with reference to Fig. 2. Fig. 2 is an explanatory diagram showing an example of an image input to the training system 1 in this embodiment.

[0067] As shown in FIG. 2, an image 20 input to the training system 1 is an image showing a scene of a person performing a task.

[0068] Specifically, image 20 is an image showing a person standing in a system kitchen in a house. There is a shelf with a door near the ceiling of the system kitchen. A bag is placed on the worktop of the system kitchen. The person is about to open the door of the shelf near the ceiling of the kitchen and store an item on the shelf.

[0069] For example, when a detection process by object detection task A is performed on image 20, a person, a shelf, a door, etc. can be detected. When an estimation process by posture estimation task B is performed on image 20, it can be estimated that a person is standing or stretching, etc. When a determination process by attribute determination task C is performed on image 20, it can be determined that the location is a kitchen, etc.

[0070] When the processing results of the above tasks A, B, and C are integrated, integrated information is generated that includes, for example, a set of tasks A, B, and C, a situation in which "a person is stretching to get something out of the kitchen," and a score of 90 associated with that situation. Also, when the processing results of the above tasks A and B are integrated, integrated information is generated that includes a set of tasks A and B, a situation in which "a person is standing up to put something away on a shelf," and a score of 60 associated with that situation.

[0071] The task selection correct answer generating unit 104 is a functional unit that generates correct answer data (hereinafter, also referred to as task selection correct answer data or, as a higher-level concept, correct answer data of model selection information) used for training the task selection model. The task selection correct answer data is correct answer data (reference data) for training the task selection model so that a task to be executed for the sensing data 10 input to the feature amount extracting unit 101 is selected from among a plurality of tasks related to the model held by the model library 30. The task selection correct answer generating unit 104 acquires integrated information from the inference control unit 102, and generates task selection correct answer data based on the acquired integrated information. Specifically, the task selection correct answer generating unit 104 generates task selection correct answer data based on a task set and a score included in the integrated information. When the integrated information includes a plurality of situations, a task set and a score associated with the same situation (hereinafter, also referred to as a set of a task set and a score) are acquired from each piece of integrated information, and task selection correct answer data is generated based on the acquired plurality of sets of task sets and scores. When the integrated information includes a single situation, task selection correct answer data is generated based on the combination of the task set and the score included in each of the multiple integrated information.

[0072] The task selection supervised answer data is generated, for example, such that a task inference model that executes a task that contributed to at least one second inference result is preferentially included in at least one task inference model indicated by the training model selection information. More specifically, the task selection supervised answer data is generated such that a task inference model that executes a task that belongs to a task set in which a score based on an inference result of at least one task inference model is higher than a threshold is preferentially included in the at least one task inference model.

[0073] Fig. 3 is an explanatory diagram showing an example of a set of a task set and a score in this embodiment. Fig. 3 illustrates a case where a set of a task set and a score associated with a specific situation is obtained.

[0074] The top set shown in Figure 3 shows task sets A, B, and C, and a score 90 for the situation obtained by integrating the inference results output by inputting a feature map into a task inference model that executes each of the tasks included in the task set, in the integration unit 103.

[0075] Similarly, the second set from the top shown in FIG. 3 shows task sets A and B and a score of 85 for the situation for that task set.

[0076] Similarly, the third set from the top shown in FIG. 3 shows task set A and a score of 60 for a given situation for that task set.

[0077] The task selection answer generation unit 104 selects a pair from among the multiple task set and score pairs described above in accordance with the magnitude of the score (i.e., the degree of contribution to the second inference result), and generates task selection answer data based on the task set of the selected pair.

[0078] For example, the top and second-top task sets shown in FIG. 3, which have scores of 80 or more, are selected, and task selection correct answer data is generated indicating that common tasks A and B are the tasks that should be selected.

[0079] Furthermore, when the integrated information includes a plurality of situations, task selection correct answer data may be generated based on a set of task set scores associated with a situation that matches a situation (i.e., correct answer data for a situation) predefined for the sensing data 10. For example, when the inference results of tasks A, B, and C are integrated to obtain a combination of a situation "a person stretching to take something out of the kitchen" and a score of 90, and the inference results of tasks A and B are integrated to obtain a combination of a situation "a person standing up to put something away on a shelf" and a score of 60, when the situation predefined for the sensing data 10 is "a person stretching to take something out of the kitchen", a set of a task set including tasks A, B, and C and a score of 90 is obtained.

[0080] The task selection correct answer data is generated according to the output format of the task selection model, in other words, the task selection correct answer data is generated according to the format of the model selection information output by the task selection model.

[0081] The model selection information is information indicating at least one task-specific inference model to which a feature map is input to execute an inference process. The model selection information is information for identifying a task-specific inference model, and includes first information corresponding to the task. For example, the first information may be information indicating a task, and a model that executes the task indicated by the first information is identified.

[0082] The task selection correct answer generating unit 104 provides a pair of the generated task selection correct answer data and sensing data to the task selection model training unit 105 as training data.

[0083] Fig. 4 is an explanatory diagram showing a first example of training data in this embodiment. The training data shown in Fig. 4 includes sensing data and task selection answer data generated based on the task sets and scores shown in Fig. 3.

[0084] 4, the training data includes sensing data and task selection answer data, which is answer data of the model selection information, in association with each other. The sensing data is, for example, an image 20 input to the training system 1, and is sensing data used when the answer data of the model selection information is generated. The answer data of the model selection information is answer data generated by the task selection answer generating unit 104 using the sensing data.

[0085] Specifically, Fig. 4 includes an image 20 as sensing data and information "A:1 B:1 C:0" as correct answer data of the model selection information in association with each other. This means that the tasks for processing the image 20 include tasks A and B, but do not include task C. Here, the model selection information is expressed in a format in which a task included in the tasks for processing the image 20 is represented by "1", and a task not included in the tasks for processing the image 20 is represented by "0". Naturally, the format of the model selection information is not limited to this.

[0086] The task selection model training unit 105 is a functional unit that trains the task selection model M1. The task selection model training unit 105 acquires training data generated by the task selection answer generation unit 104, and also acquires a feature map from the feature extraction unit 101 as a feature of sensing data included in the training data. The task selection model training unit 105 inputs the acquired feature map to the task selection model M1 to acquire model selection information as model selection information for training. Then, the task selection model training unit 105 trains the task selection model M1 by machine learning using the task selection answer data as reference data and the model selection information as output data, and outputs the trained task selection model M1.

[0087] The task selection model M1 is a model trained using machine learning based on at least one inference result obtained by inputting a feature map into at least one task inference model among a plurality of task inference models held in the model library 30, and model selection information obtained by inputting a feature map into the task selection model M1. The task selection model M1 is, for example, a neural network model, and is an example of a third inference model.

[0088] 4, the task selection model training unit 105 trains the task selection model M1 such that when sensing data included in the training data is input to the training system 1, the task selection model M1 outputs correct answer data of the model selection information associated with the sensing data in the training data. For example, the task selection model M1 can be trained by calculating, by a loss function, the difference between the model selection information output when a feature map extracted from the sensing data by the feature extraction unit 101 is input to the task selection model M1 and the correct answer data of the model selection information included in the training data, and adjusting the parameters included in the task selection model M1 so as to reduce the difference.

[0089] The task selection model M1 trained by the training system 1 as described above is provided to the inference system 2 and used for inference processing of sensing data.

[0090] The processing executed by the training system 1 configured as above will be described below.

[0091] FIG. 5 is a flow diagram showing the processing executed by training system 1 in this embodiment.

[0092] In step S1001, the feature extraction unit 101 acquires the sensing data 10.

[0093] In step S1002, the feature extraction unit 101 performs a process of extracting features of the sensing data 10 acquired in step S1001, thereby acquiring a feature map of the sensing data 10.

[0094] In step S1003, the inference control unit 102 determines a task set for executing inference processing on the feature map acquired in step S1002. At this time, the inference control unit 102 determines a plurality of task sets, at least some of which are different from each other.

[0095] In step S1004, the inference control unit 102 executes an inference process using a model that executes a task belonging to each of the multiple task sets determined in step S1003.

[0096] In step S1005, the integration unit 103 generates integrated information by integrating inference results indicating the results of the inference processes executed in step S1004 using models that execute tasks belonging to each of the multiple task sets.

[0097] In step S1006, the task selection correct answer generating unit 104 generates task selection correct answer data based on the integrated information generated in step S1005.

[0098] In step S1007, the task selection model training unit 105 trains the task selection model M1 by performing machine learning using the task selection correct answer data generated in step S1006.

[0099] 5, the training system 1 trains the task selection model M1. The training system 1 provides the trained task selection model M1 to the inference system 2.

[0100] (1-2) Inference System 2 Next, a description will be given of the inference system 2. Note that a description of configurations that are substantially the same as those in the training system 1 will be omitted.

[0101] FIG. 6 is a block diagram showing the functional configuration of the inference system 2 in this embodiment.

[0102] As shown in FIG. 6, the inference system 2 selects an inference model based on the input sensing data 10, and performs inference processing on the sensing data 10 using the selected inference model.

[0103] As shown in FIG. 6, the inference system 2 includes a feature extraction unit 201, an inference control unit 202, a task selection unit 208, a model selection unit 210, and a model library 30.

[0104] The feature extraction unit 201 is a functional unit that extracts features of the sensing data 10 using the feature extraction model L1. The feature extraction unit 201 is the same functional unit as the feature extraction unit 101 provided in the training system 1. The feature extraction unit 201 generates a feature map indicating the features of the input sensing data 10, and provides the generated feature map to the inference control unit 202 and the task selection unit 208.

[0105] The inference control unit 202 is a functional unit that acquires a feature map from the feature amount extraction unit 201, and executes an inference process on the acquired feature map using a task inference model selected by the model selection unit 210. The inference control unit 202 outputs result information 50 indicating a result of the inference process. Before executing the inference process, the inference control unit 202 operates the task inference model selected by the model selection unit 210, and executes the inference process using the operated task inference model. When executing the inference process, the inference control unit 202 operates only the task inference model selected by the model selection unit 210 among the task inference models held. Task inference models not selected by the model selection unit 210 are not operated. The result information 50 corresponds to a second inference result.

[0106] The task selection unit 208 is a functional unit that selects a task of a task inference model that the inference control unit 202 uses to execute the inference process. The task selection unit 208 acquires and holds a task selection model M1 trained by the training system 1. The task selection unit 208 acquires a feature map from the feature extraction unit 201, and uses the task selection model M1 to generate model selection information that identifies a model that executes a task to be executed for the feature map, and provides the model selection information to the model selection unit 210.

[0107] The model selection unit 210 is a functional unit that selects an inference model for a task that the inference control unit 202 uses to execute the inference process.

[0108] The model selection unit 210 acquires the model selection information generated by the task selection unit 208. Then, the model selection unit 210 selects an inference model for a task from the model library 30 in accordance with the acquired model selection information, and provides the model to the inference control unit 202. Specifically, the model selection unit 210 identifies a task from the first information included in the model selection information, and selects an inference model for a task that executes the identified task.

[0109] The model selection unit 210 may further acquire device information 40 of the hardware on which the inference system 2 is running, particularly the hardware on which the inference control unit 202 is running, and may select a task inference model from the model library 30 by considering the acquired device information 40 together with the model selection information. The device information 40 is information including the amount of static computing resources, the current or future availability of computing resources, and the like. For example, if the specifications of the CPU or memory of the hardware on which the inference control unit 202 is running are relatively low, a task inference model having a performance lower than that of the task inference model indicated in the model selection information may be acquired from the model library 30.

[0110] When the model selection unit 210 selects a task inference model taking into consideration the device information 40, it determines an upper limit of the computational complexity of an operable task inference model based on information indicating the performance of the task inference model as an operating condition and the device information 40. Then, the model selection unit 210 selects a task inference model whose computational complexity is equal to or less than the upper limit from among task inference models that execute a task specified according to the model selection information.

[0111] The model library 30 is a library that holds inference models for tasks that the inference control unit 202 can use to execute inference processing.

[0112] The feature extraction unit 201 corresponds to a first acquisition unit that inputs sensing data to a first inference model and acquires a first inference result for the sensing data from the first inference model.

[0113] The task selection unit 208 corresponds to a second acquisition unit that acquires model selection information indicating at least one second inference model selected from among multiple second inference models from the third inference model by inputting the first inference result into the third inference model.

[0114] The inference control unit 202 corresponds to a third acquisition unit that acquires at least one second inference result for the first inference result from the second inference model by inputting the first inference result to at least one second inference model indicated by the model selection information, and an output unit that outputs at least one second inference result.

[0115] The processing executed by the inference system 2 configured as above will be explained below.

[0116] FIG. 7 is a flow diagram showing the processing executed by the inference system 2 in the embodiment.

[0117] In step S2001, the feature extraction unit 201 acquires the sensing data 10.

[0118] In step S2002, the feature extraction unit 201 performs a process of extracting features of the sensing data 10 acquired in step S2001, thereby acquiring a feature map of the sensing data 10.

[0119] In step S2003, the task selection unit 208 acquires the feature map acquired in step S2002, and acquires model selection information that is output by inputting the acquired feature map to the task selection model M1.

[0120] In step S2004, the model selection unit 210 selects the task inference model indicated by the model selection information generated in step S2003 from among the task inference models held in the model library 30, and provides the task inference model to the inference control unit 102.

[0121] In step S2005, the inference control unit 202 executes an inference process and acquires an inference result by inputting the feature map provided by the feature extraction unit 201 into the task inference model provided by the model selection unit 210. The inference control unit 202 outputs the acquired inference result.

[0122] 7, the inference system 2 can execute the inference process by operating an inference model suitable for the inference process on the sensing data 10. At this time, the inference system 2 does not operate any inference model other than the inference model suitable for the inference process on the sensing data 10, so that it is possible to reduce the computational resources used or the power consumption.

[0123] (Embodiment 2) In this embodiment, a performance selection model is trained instead of the task selection model in the first embodiment, and an inference model to be used for inference processing on sensing data is selected using the output result of the performance selection model. Note that the description of the configuration that is substantially the same as the configuration of the first embodiment will be omitted.

[0124] (2-1) Training System 1 FIG. 8 is a block diagram showing the configuration of a training system 1 which is a processing system in this embodiment.

[0125] As shown in FIG. 8, the training system 1 outputs an inference model (performance selection model M2) that selects an inference model for executing inference processing on the sensing data 10 based on the input sensing data 10.

[0126] The training system 1 includes a feature extraction unit 301 , an inference control unit 302 , an integration unit 303 , a performance selection answer generation unit 306 , a performance selection model training unit 307 , and a model library 30 .

[0127] The feature extraction unit 301 generates a feature map based on the sensing data 10 using the feature extraction model L3.

[0128] The inference control unit 302 controls the inference process using the task inference models managed in the model library 30 based on the generated feature map. The task inference models held in the model library 30 include task inference models that execute various tasks and also include models with various performances. For example, task inference models that execute the same task and have different performances are included. For example, "model A1" is a model that executes task A and has relatively low performance. Also, "model B2" is a model that executes task B and has relatively medium performance. Here, the performance of the task inference models is expressed in three stages: 1 (low), 2 (medium), and 3 (high), but is not limited to this.

[0129] The inference control unit 302 inputs a feature map to at least one of the task inference models with different performances and executes the inference process. For example, the inference control unit 302 inputs a feature map to each of the task inference models A1 to A3, B1 to B3, and C1 to C3 that execute the task of object detection, and acquires an inference result. The inference results may differ, for example, in the position, size, likelihood, etc. of the object. Note that the inference process may be executed for only one specific performance (for example, only a task inference model with medium performance) among the task inference models with different performances, or the inference process may be executed for all of the task inference models with different performances.

[0130] The integration unit 303 generates integrated information based on one or more inference results for each task. In this embodiment, unlike the first embodiment, a task is not selected, so the integrated information includes a set of a situation and a score. The integration unit 303 selects one of the inference results of the task-specific inference models with different capabilities for each task, and generates integrated information based on the selected inference result. The generation of integrated information is the same as in the first embodiment. Note that integrated information may be generated for only one specific capability among the inference results of the task-specific inference models with different capabilities, or integrated information may be generated for all of the inference results of the task-specific inference models with different capabilities. In the latter case, the integrated information includes a combination of capabilities (hereinafter, a performance set). In addition, when there is one task, the integrated information may include the inference result instead of the situation.

[0131] The performance selection answer generation unit 306 is a functional unit that generates answer data (hereinafter, also referred to as performance selection answer data or, as a higher-level concept, answer data of model selection information) used for training the performance selection model M2. The performance selection answer data is answer data (reference data) for training the performance selection model M2 so that a model having appropriate performance as a model for executing inference processing for the sensing data 10 input to the feature extraction unit 301 is selected. The performance selection answer generation unit 306 acquires integrated information from the integration unit 303, and generates data specifying performance required for inference processing as performance selection answer data based on the acquired integrated information.

[0132] Specifically, the performance selection correct answer generating unit 306 generates performance selection correct answer data that specifies lower performance as the integrated information is better, and generates performance selection correct answer data that specifies higher performance as the integrated information is worse. Evaluation of whether the integrated information is good or not may be performed based on a comparison with correct answer data of the integrated information (e.g., correct answer data defined in advance for the sensing data 10), or may be performed based on the contents of the integrated information. Details will be described later. The performance selection correct answer generating unit 306 provides the generated performance selection correct answer data to the performance selection model training unit 307.

[0133] The performance selection answer data is generated, for example, such that a task-specific inference model that satisfies a performance requirement in at least one second inference result is preferentially included in at least one task-specific inference model indicated by the training model selection information. More specifically, the performance selection answer data is generated such that a task-specific inference model having a performance according to the difficulty of processing of the task-specific inference model is preferentially included in at least one task-specific inference model indicated by the training model selection information.

[0134] The performance selection answer data is generated according to a predetermined format output by the performance selection model M2. In other words, the performance selection answer data is generated according to the format of the model selection information output by the performance selection model M2.

[0135] The model selection information is information indicating at least one task-specific inference model to which a feature map is input to execute inference processing. The model selection information is information for identifying the task-specific inference model, and includes second information corresponding to the performance of the task-specific inference model. The second information is, for example, information indicating the performance of the task-specific inference model. Specifically, the information indicating the performance may be information indicating the processing accuracy of the task-specific inference model, or the processing speed of the model, or at least one of these scores. The information indicating the performance may also include size information indicating the size of the neural network model. The size information may include the number of nodes, the number of branches, or the number of layers of the neural network model.

[0136] Furthermore, the second information may be, for example, information indicating the difficulty of processing the inference model for the task. According to the difficulty, a model selection unit 410 described later selects an inference model for the task with performance corresponding to the difficulty. The information indicating the difficulty is information indicating the degree of difficulty or ease of processing. Here, "difficult processing" means that the accuracy required for the processing is high, or the amount of calculation required for the processing is large. The difficulty may be expressed as a discrete value (0 or 1, etc.) or a continuous value (0 to 1, etc.).

[0137] Here, the generation of the performance selection supervised data will be described in detail. Note that, when the integration information includes an inference result, the integration process is interpreted as being replaced with an inference process.

[0138] For example, if the integration process is a process for solving a classification problem, the difficulty level may be set to high when the classification is incorrect (i.e., the classification result does not match the ground truth data), or when at least some of the likelihoods of multiple classes are close to each other. Note that in the latter case, the ground truth data is not used.

[0139] In addition, when the integration process is a process of solving a regression problem, the difficulty level may be set according to the difference between the integration information and the correct answer data, or the difference may be set as the difficulty level as it is. In addition, the difficulty level may be set to be high when the difference exceeds a certain threshold. The difference may be calculated simply by subtraction, or may be calculated by any function that uses the subtraction result as an input.

[0140] Furthermore, when the integration process is an object detection problem, that is, a process of solving a combination problem of a classification problem (a problem of classifying a frame) for a certain detection frame and a regression problem (a problem of determining the coordinates of the frame), the difficulty level may be a combination of the setting methods for the above-mentioned "classification problem" and "regression problem". In this case, the difficulty level of the regression problem part may be set to a value (1-IoU) obtained by subtracting the overlap rate (IoU: Intersection over Union) between the correct answer frame and the detection frame from 1. Furthermore, the difficulty level of the object detection problem may be a linear combination or multiplication of the difficulty level of the classification problem and the difficulty level of the regression problem.

[0141] In addition, when there are multiple pieces of integrated information, the performance selection correct answer generating unit 306 may calculate the difficulty of each piece of integrated information, and integrate the calculated difficulty levels to set one difficulty level. For example, the number of times when the situation included in the integrated information matches a situation (i.e., correct answer data for the situation) defined in advance for the sensing data 10 may be counted, and the difficulty level may be set according to the magnitude of the count number. For example, the performance selection correct answer data may be generated that indicates that the greater the count number (i.e., the greater the number of matches), the lower the difficulty level (i.e., the lower the performance). Also, the performance selection correct answer data may be generated that indicates that the smaller the count number (i.e., the fewer the number of matches), the higher the difficulty level (i.e., the higher the performance).

[0142] The performance selection answer generating unit 306 provides a pair of the generated performance selection answer data and sensing data to the performance selection model training unit 307 as training data.

[0143] Fig. 9 is an explanatory diagram showing an example of training data in this embodiment. The training data shown in Fig. 9 includes sensing data and performance selection answer data generated based on an inference result.

[0144] Specifically, FIG. 9 includes an image 20 as sensing data and information "3" as correct answer data in the model selection information in association with each other. This means that the performance of the task-specific inference model that processes the image 20 is required to be "high." Here, the model selection information is expressed in a format that expresses the performance (or difficulty) required of the task-specific inference model that processes the image 20 in three stages: 1 (low), 2 (medium), and 3 (high). Naturally, the format of the model selection information is not limited to this.

[0145] The performance selection model training unit 307 is a functional unit that generates the performance selection model M2. The performance selection model training unit 307 acquires the performance selection correct answer data generated by the performance selection correct answer generating unit 306, and also acquires a feature map from the feature amount extracting unit 301 as a feature amount of the sensing data included in the training data. The performance selection model training unit 307 inputs the acquired feature map to the performance selection model M2 to acquire model selection information as training model selection information. Then, the performance selection model training unit 307 trains the performance selection model M2 by machine learning using the performance selection correct answer data as reference data and the model selection information as output data, and outputs it. The performance selection model M2 is an example of a third inference model.

[0146] 9, the performance selection model training unit 307 trains the performance selection model M2 so that when sensing data included in the training data is input to the training system 1, the performance selection model M2 outputs correct answer data of the model selection information associated with the sensing data in the training data. The training of the performance selection model M2 can be performed by calculating the difference between the model selection information output when the feature map extracted from the sensing data by the feature extraction unit 301 is input to the performance selection model M2 and the correct answer data of the model selection information included in the training data by using a loss function, and adjusting the parameters included in the performance selection model M2 so as to reduce the difference.

[0147] The performance selection model M2 trained by the training system 1 as described above is provided to the inference system 2 and used for inference processing of the sensing data.

[0148] The processing executed by the training system 1 configured as above will be described below.

[0149] FIG. 10 is a flow diagram showing the processing executed by the training system 1 in this embodiment.

[0150] In step S3001, the feature extraction unit 301 acquires the sensing data 10.

[0151] In step S3002, the feature extraction unit 301 performs a process of extracting the feature of the sensing data 10 acquired in step S3001, thereby acquiring a feature map of the sensing data 10.

[0152] In step S3003, the inference control unit 302 inputs the sensing data 10 to task-specific inference models having different performances held in the model library 30, and executes inference processing.

[0153] In step S3004, the performance selection answer generating unit 306 generates performance selection answer data based on the inference result output in step S3003.

[0154] In step S3005, the performance selection model training unit 307 trains the performance selection model M2 by performing machine learning using the performance selection correct answer data generated in step S3004.

[0155] 10, the training system 1 trains the performance selection model M2. The training system 1 provides the trained performance selection model M2 to the inference system 2.

[0156] (2-2) Inference System 2 Next, the inference system 2 will be described.

[0157] FIG. 11 is a block diagram showing the functional configuration of the inference system 2 in this embodiment.

[0158] As shown in FIG. 11, the inference system 2 includes a feature extraction unit 401, an inference control unit 402, a performance selection unit 409, a model selection unit 410, and a model library 30.

[0159] The feature extraction unit 401 generates a feature map indicating the features of the input sensing data 10 using the feature extraction model L3, and provides the generated feature map to the inference control unit 402 and the performance selection unit 409.

[0160] The performance selection unit 409 is a functional unit that selects the performance of a task inference model used by the inference control unit 402 to execute inference processing. The performance selection unit 409 acquires and holds the performance selection model M2 trained by the training system 1. The performance selection unit 409 acquires a feature map from the feature extraction unit 401, and uses the performance selection model M2 to generate model selection information that specifies the performance of a task inference model to be used in inference processing for the feature map, and provides the model selection unit 410 with the model selection information.

[0161] The model selection unit 410 acquires the model selection information generated by the performance selection unit 409. Then, the model selection unit 410 selects an inference model for a task from the model library 30 in accordance with the acquired model selection information, and provides the model to the inference control unit 402. Specifically, the model selection unit 410 specifies performance from the second information included in the model selection information, and selects an inference model for a task having the specified performance.

[0162] In addition, the model selection unit 410 may further take into consideration the device information 40 in addition to the above model selection information to select an inference model for a task from the model library 30.

[0163] When the model selection unit 410 selects a task inference model taking into consideration the device information 40, it determines an upper limit of the computational complexity of an operable task inference model based on information indicating the performance of the task inference model as an operating condition and the device information 40. Then, the model selection unit 410 selects a task inference model whose computational complexity is equal to or less than the upper limit from among task inference models having performance specified according to the model selection information.

[0164] The processing executed by the inference system 2 configured as above will be explained below.

[0165] FIG. 12 is a flow diagram showing the processing executed by the inference system 2 in the embodiment.

[0166] In step S4001, the feature extraction unit 401 acquires the sensing data 10.

[0167] In step S4002, the feature extraction unit 401 performs a process of extracting the feature of the sensing data 10 acquired in step S4001, thereby acquiring a feature map of the sensing data 10.

[0168] In step S4003, the performance selection unit 409 acquires the feature map acquired in step S4002, and acquires model selection information to be output by inputting the acquired feature map to the performance selection model M2.

[0169] In step S4004, the model selection unit 410 selects the task inference model indicated by the model selection information generated in step S4003 from the task inference models held in the model library 30, and provides the task inference model to the inference control unit 402.

[0170] In step S4005, the inference control unit 402 executes inference processing and acquires an inference result by inputting the feature map provided by the feature extraction unit 401 into the task inference model provided by the model selection unit 410. The inference control unit 402 outputs the acquired inference result.

[0171] 12, the inference system 2 can execute the inference process by operating an inference model suitable for the inference process on the sensing data 10. At this time, the inference system 2 does not operate any inference model other than the inference model suitable for the inference process on the sensing data 10, so that it is possible to reduce the computational resources used or the power consumption.

[0172] (Embodiment 3) In this embodiment, both the task selection model of embodiment 1 and the performance selection model of embodiment 2 are trained, and an inference model to be used for inference processing on sensing data is selected using the output results of the task selection model and the performance selection model. Note that a description of the configuration that is substantially the same as the configuration of embodiment 1 or 2 will be omitted.

[0173] (3-1) Training System 1 FIG. 13 is a block diagram showing the configuration of a training system 1 which is a processing system in this embodiment.

[0174] As shown in FIG. 13, the training system 1 outputs inference models (task selection model M1 and performance selection model M2) that select an inference model that performs inference processing on the input sensing data 10 based on the input sensing data 10.

[0175] The training system 1 includes a feature extraction unit 501, an inference control unit 502, an integration unit 503, a task selection answer generation unit 504, a task selection model training unit 505, a performance selection answer generation unit 506, and a performance selection model training unit 507.

[0176] The feature extraction unit 501 generates a feature map based on the sensing data 10 using the feature extraction model L5.

[0177] The inference control unit 502 is a task-specific inference model that executes each of the tasks included in a task set selected from the determined multiple task sets, and inputs a feature map to at least one of the task-specific inference models with different performance to execute inference processing.

[0178] The integration unit 503 acquires integrated information based on the inference results (second inference results) output by each of the task inference models. The integrated information includes a task set, a situation estimated from the sensing data 10, and a set of scores indicating the likelihood of estimating the situation. The integrated information may also include a performance set.

[0179] The task selection correct answer generating unit 504 generates task selection correct answer data based on the generated integrated information. Also, the task selection correct answer generating unit 504 generates training data for the task selection model M1 based on the generated task selection correct answer data.

[0180] The task selection model training unit 505 trains the task selection model M1 using the generated training data.

[0181] The performance selection correct answer generating unit 506 generates performance selection correct answer data based on the generated integrated information. Specifically, the performance selection correct answer generating unit 506 generates performance selection correct answer data based on integrated information about the task set indicated by the generated task selection correct answer data. For example, when the task set indicated by the task selection correct answer data is tasks A, B, and C, when integrated information indicating performance 2 (medium) is generated, a situation that is a result of integrating the inference results of model A2, model B2, and model C2 is compared with the correct answer data of the situation. For example, when the situation does not match the correct answer data, a high-performance task inference model is required (in other words, the difficulty level is high), and performance selection correct answer data indicating 3 (high) is generated. In addition, the performance selection correct answer generating unit 506 generates training data for the performance selection model M2 based on the generated performance selection correct answer data. Note that, in the above, the integrated information used to generate the performance selection correct answer data is the integrated information about the task set indicated by the generated task selection correct answer data, but is not limited to this, and may be all the generated integrated information.

[0182] The performance selection model training unit 507 trains the performance selection model M2 using the generated training data.

[0183] The task selection model M1 and the performance selection model M2 trained by the training system 1 as described above are provided to the inference system 2 and used for inference processing of the sensing data.

[0184] The processing executed by the training system 1 configured as above will be described below.

[0185] FIG. 14 is a flow diagram showing the processing executed by training system 1 in this embodiment.

[0186] In step S5001, the feature extraction unit 501 acquires the sensing data 10.

[0187] In step S5002, the feature extraction unit 501 performs a process of extracting the feature of the sensing data 10 acquired in step S5001, thereby acquiring a feature map of the sensing data 10.

[0188] In step S5003, the inference control unit 502 determines a task set for executing inference processing on the feature map acquired in step S5002. At this time, the inference control unit 502 determines a plurality of task sets, at least some of which are different from each other.

[0189] In step S5004, the inference control unit 502 executes an inference process using a model that executes a task belonging to each of the multiple task sets determined in step S5003.

[0190] In step S5005, the integration unit 503 generates integrated information by integrating inference results indicating the results of the inference processes executed in step S5004 using models that execute tasks belonging to each of the multiple task sets.

[0191] In step S5006, the task selection correct answer generating unit 504 generates task selection correct answer data based on the integrated information generated in step S5005.

[0192] In step S5007, the task selection model training unit 105 trains the task selection model M1 by performing machine learning using the task selection correct answer data generated in step S5006.

[0193] In step S5008, the performance selection correct generating unit 506 generates performance selection correct data based on the integrated information generated in step S5005 and the task selection correct data generated in step S5006.

[0194] In step S5009, the performance selection model training unit 507 trains the performance selection model M2 by performing machine learning using the performance selection correct data generated in step S5008.

[0195] 14, the training system 1 trains the task selection model M1 and the performance selection model M2. The training system 1 provides the trained task selection model M1 and the performance selection model M2 to the inference system 2.

[0196] (3-2) Inference System 2 Next, a description will be given of the inference system 2. Note that a description of configurations that are substantially the same as those in the training system 1 will be omitted.

[0197] FIG. 15 is a block diagram showing the functional configuration of the inference system 2 in this embodiment.

[0198] As shown in FIG. 15, the inference system 2 includes a feature extraction unit 601, an inference control unit 602, a task selection unit 608, a performance selection unit 609, a model selection unit 610, and a model library 30.

[0199] The feature extraction unit 601 generates a feature map indicating the characteristics of the input sensing data 10 using the feature extraction model L5, and provides the generated feature map to the inference control unit 602, the task selection unit 608, and the performance selection unit 609.

[0200] The model selection unit 610 acquires model selection information (hereinafter, model selection information 1) generated by the task selection unit 608 and model selection information (hereinafter, model selection information 2) generated by the performance selection unit 609. Then, the model selection unit 610 selects a task inference model from the model library 30 according to the two acquired model selection information, and provides it to the inference control unit 602. Specifically, the model selection unit 610 identifies a task from the first information included in the model selection information 1, and identifies performance from the second information included in the model selection information 2. The model selection unit 610 selects a task inference model with the identified performance that executes the identified task. Note that there may be only one model selection information, and in this case, the one model selection information may include the first information and the second information.

[0201] In addition, the model selection unit 610 may further take into consideration the device information 40 in addition to the above two pieces of model selection information to select an inference model for a task from the model library 30.

[0202] When the model selection unit 610 selects a task inference model taking into consideration the device information 40, it determines an upper limit of the computational complexity of an operable task inference model based on information indicating the performance of the task inference model as an operating condition and the device information 40. Then, the model selection unit 610 selects a task inference model whose computational complexity is equal to or less than the upper limit from among task inference models that execute a task specified according to the model selection information 1 and have performance specified according to the model selection information 2.

[0203] When the model selection unit 610 selects an inference model for a task, there may be various patterns as to which of the model selection information 1, the model selection information 2, and the device information 40 should be prioritized. For example, the inference model for a task may be selected by prioritizing the device information 40, the model selection information 1, and the model selection information 2 in that order.

[0204] However, the order of priority is not limited to the above, and may be, for example, the model selection information 2, the model selection information 1, and the device information 40, or the model selection information 1, the model selection information 2, and the device information 40.

[0205] The processing executed by the inference system 2 configured as above will be explained below.

[0206] FIG. 16 is a flow diagram showing the processing executed by the inference system 2 in the embodiment.

[0207] In step S6001, the feature extraction unit 601 acquires the sensing data 10.

[0208] In step S6002, the feature extraction unit 601 performs a process of extracting the feature of the sensing data 10 acquired in step S6001, thereby acquiring a feature map of the sensing data 10.

[0209] In step S6003, the task selection unit 608 acquires the feature map acquired in step S6002, and acquires model selection information 1 that is output by inputting the acquired feature map to the task selection model M1.

[0210] In step S6004, the performance selection unit 609 acquires the feature map acquired in step S6002, and acquires model selection information 2 that is output by inputting the acquired feature map to the performance selection model M2.

[0211] In step S6005, the model selection unit 610 selects an inference model for the task indicated by the model selection information 1 generated in step S6003 and the model selection information 2 generated in step S6004 from among the inference models for the task held in the model library 30, and provides the selected model to the inference control unit 602.

[0212] In step S6006, the inference control unit 602 executes inference processing by inputting the feature map provided by the feature extraction unit 601 to the task inference model provided by the model selection unit 610, and acquires an inference result. The inference control unit 602 outputs the acquired inference result.

[0213] The processing in steps S6003 and S6004 may be executed in the opposite order, or may be executed simultaneously in parallel.

[0214] 16, the inference system 2 can execute the inference process by operating an inference model suitable for the inference process on the sensing data 10. At this time, the inference system 2 does not operate any inference model other than the inference model suitable for the inference process on the sensing data 10, so that it is possible to reduce the computational resources used or the power consumption.

[0215] (Embodiment 4) In this embodiment, instead of the task selection model and the performance selection model, an inference model to be used for inference processing on sensing data is selected using the output result of the feature extraction model. Note that, in this embodiment, an example in which an inference model is selected from the viewpoints of both task and performance will be described, but an inference model may be selected from the viewpoints of only task or only performance as in the first or second embodiment.

[0216] (4-1) Training System 1 FIG. 17 is a block diagram showing the configuration of a training system 1 which is a processing system in this embodiment.

[0217] As shown in FIG. 17, the training system 1 includes a feature extractor 701, an inference controller 702, an integration unit 703, a task selection answer generator 704, a performance selection answer generator 706, and a training unit 711.

[0218] Training system 1 in this embodiment differs from training system 1 in embodiment 3 in that it does not include task selection model training unit 505 and performance selection model training unit 507 in embodiment 3, but includes a training unit 711 instead.

[0219] The feature extraction unit 701 is the same as in each embodiment in that it extracts features based on sensing data, but the feature extraction model used to extract the features is different. The feature extraction model M3 in this embodiment receives sensing data as input and outputs features and model selection information. Details of the feature extraction model M3 will be described later.

[0220] The following description will focus on the training section 711.

[0221] The training unit 711 is a functional unit that trains the feature extraction model M3. The training unit 711 acquires the task selection correct answer data generated by the task selection correct answer generation unit 704 and the performance selection correct answer data generated by the performance selection correct answer generation unit 706, and also acquires the sensing data 10. The training unit 711 inputs the acquired sensing data 10 to the feature extraction model M3 to acquire model selection information as training model selection information. Then, the training unit 711 trains the feature extraction model M3 by machine learning using the task selection correct answer data and the performance selection correct answer data as reference data and the model selection information as output data, and outputs the trained model. Here, the task selection correct answer data and the performance selection correct answer data are the same as those in the third embodiment.

[0222] The model selection information in this embodiment includes first information and second information. As described above, the first information is information corresponding to a task, and is information for identifying a task or information for identifying a task-use inference model that executes a task. The second information is information corresponding to performance, and is information for identifying performance or information for identifying a model having performance.

[0223] The feature extraction model M3 is a model trained using machine learning based on at least one inference result (second inference result) obtained by inputting a feature map output by the feature extraction model M3 into at least one task inference model among a plurality of task inference models held in the model library 30, and model selection information obtained by inputting the sensing data 10 into the feature extraction model M3. Note that the feature extraction model M3 is an example of a first inference model.

[0224] The feature extraction model M3 outputs features and model selection information when the sensing data 10 is input. Specifically, the feature extraction model M3 includes a part that extracts features, and a part that outputs model selection information when the extracted features are input. For example, the part that outputs the model selection information may be input with the same features as those output outside the model, or may be input with a feature that is an intermediate product that is not output outside the model (for example, a feature output by an intermediate layer, not the final layer of the part that extracts features).

[0225] In training the feature extraction model M3 using the task selection correct answer data and the performance selection correct answer data, the training unit 711 does not train the part that extracts features, but trains only the part that outputs model selection information. For example, the training unit 711 fixes the parameters of the part that extracts features, and updates the parameters only of the part that outputs model selection information. When training the part that extracts features, if the part that outputs model selection information is trained at the same time, the parameters of both may be updated as training targets.

[0226] The processing executed by the training system 1 configured as above will be described below.

[0227] FIG. 18 is a flow diagram showing the processing executed by the training system 1 in this embodiment.

[0228] Among the processes executed by training system 1, those parts that differ from the processes executed by training system 1 of the above-mentioned third embodiment will be mainly described.

[0229] The processing from steps S7001 to S7007 shown in FIG. 18 is the same as the processing from steps S5001 to S5006 and the processing of step S5008 (see FIG. 14) executed by the training system 1 of the above-mentioned third embodiment.

[0230] In step S7008, the training unit 711 trains the feature extraction model M3 by performing machine learning using the task selection correct answer data generated in step S7006 and the performance selection correct answer data generated in step S7007.

[0231] 18, the training system 1 trains the feature extraction model M3. The training system 1 provides the trained feature extraction model M3 to the inference system 2.

[0232] (4-2) Inference System 2 FIG. 19 is a block diagram showing the functional configuration of the inference system 2 in this embodiment.

[0233] As shown in FIG. 19, the inference system 2 includes a feature extraction unit 801, an inference control unit 802, a model selection unit 810, and a model library 30.

[0234] The feature extraction unit 801 is a functional unit that outputs features and model selection information based on the sensing data 10. Specifically, the feature extraction unit 801 generates features and model selection information by inputting the sensing data 10 to a feature extraction model M3. The model selection information includes information for identifying a task and performance. The feature extraction unit 801 provides the generated features (feature map) to the inference control unit 802, and provides the generated model selection information to the model selection unit 810. Therefore, the feature extraction unit 801 can be said to be a functional unit that combines the functions of the feature extraction unit 601, task selection unit 608, and performance selection unit 609 of the third embodiment.

[0235] Similar to the inference control unit 802 of embodiment 3, the inference control unit 802 acquires a feature map from the feature extraction unit 801, executes inference processing by inputting the acquired feature map into the inference model for the task selected by the model selection unit 810, and outputs result information 50 (corresponding to the second inference result).

[0236] In addition, the feature extraction unit 801 corresponds to a first acquisition unit that acquires a first inference result for the sensing data and model selection information indicating at least one second inference model selected from a plurality of second inference models from the first inference model by inputting the sensing data into the first inference model.

[0237] The inference control unit 802 corresponds to a second acquisition unit that acquires at least one second inference result for the first inference result from the second inference model by inputting the first inference result to at least one second inference model indicated by the model selection information, and an output unit that outputs at least one second inference result.

[0238] The processing executed by the inference system 2 configured as above will be explained below.

[0239] FIG. 20 is a flow diagram showing the processing executed by the inference system 2 in this embodiment.

[0240] The processing executed by the inference system 2 will be mainly explained with reference to the parts that are different from the processing executed by the inference system 2 of the third embodiment.

[0241] The process of step S8001 shown in FIG. 20 is the same as the process of step S6001 (see FIG. 16) executed by the inference system 2 of the third embodiment.

[0242] In step S8002, the feature extraction unit 801 inputs the sensing data 10 acquired in step S8001 to the feature extraction model M3 to generate a feature map and model selection information.

[0243] In step S8003, the model selection unit 810 selects the task inference model indicated by the model selection information generated in step S8003 from the task inference models held in the model library 30, and provides the task inference model to the inference control unit 802.

[0244] In step S8004, the inference control unit 802 executes inference processing and acquires an inference result by inputting the feature map provided by the feature extraction unit 801 into the task inference model provided by the model selection unit 810. The inference control unit 802 outputs the acquired inference result.

[0245] 20, the inference system 2 can execute the inference process by operating an inference model suitable for the inference process on the sensing data 10. Therefore, the inference system 2 of this embodiment has the same effects as the inference system 2 of the above-mentioned embodiment 3.

[0246] (summary) As described above, according to the information processing method according to the first to fourth embodiments, the second inference result is obtained using at least one second inference model selected from a plurality of second inference models, so that when the second inference result is obtained, the second inference model that is not selected from the plurality of second inference models is not operated. Therefore, the computational resources or power required for the operation of the second inference model that is not selected can be reduced. In this way, the information processing method can suppress the computational cost required for the inference process while enabling inference processing for various inference targets. In addition, since the second inference model is selected based on the first inference result obtained by inputting sensing data, the second inference model is substantially selected based on the characteristics of the sensing data. Therefore, it is possible to improve the possibility that an inference model suitable for the data to be inferred can be selected.

[0247] In addition, by using machine learning, it is possible to generate a third inference model that performs the complex inference process of selecting a second inference model from the characteristics of the sensing data more easily than with other methods.

[0248] Furthermore, by generating data conforming to the format of the model selection information from the second inference result for training, it is possible to train the third inference model without being restricted by the format of the second inference result for training. In other words, various second inference models that output various second inference results can be applied to the embodiments of the present disclosure.

[0249] In addition, since the second inference model that executes a task that contributed to at least one second inference result is preferentially included in training model selection information as reference data for machine learning, it is possible to automatically generate a third inference model that outputs model selection information for selecting the second inference model that executes the task that contributed to the second inference result. In addition, the reference data contains more data that is effective for training the third inference model, improving the efficiency of training.

[0250] In addition, since the second inference model that executes tasks belonging to a task set whose score is higher than the threshold is preferentially included in the training model selection information as reference data during machine learning, the level of precision or accuracy of the second inference result by the second inference model that is actually selected by the third inference model can be brought closer to a predetermined standard.

[0251] In addition, since the second inference model that satisfies the performance requirements is preferentially included in the training model selection information as reference data for machine learning, it is possible to automatically generate a third inference model that outputs model selection information for selecting the second inference model that satisfies the performance requirements. In addition, the reference data contains more data that is effective for training the third inference model, improving the efficiency of training.

[0252] In addition, since the second inference model having performance according to the level of difficulty is preferentially included in the training model selection information used as reference data during machine learning, the processing performance (in other words, the computational cost) of the inference processing by the second inference model that is actually selected by the third inference model can be brought closer to the required level.

[0253] Furthermore, since the size information of the neural network is used as the performance of the second inference model, the performance of the second inference model can be adjusted more easily.

[0254] In addition, since the second inference result is obtained using at least one second inference model selected from the plurality of second inference models, when the second inference result is obtained, the second inference model that is not selected from the plurality of second inference models is not operated. Therefore, the computational resources or power required for the operation of the second inference model that is not selected can be reduced. In this way, the above information processing method can suppress the computational costs of the inference process while enabling inference processing for various inference targets. In addition, since the second inference model is selected substantially based on the sensing data, it is possible to improve the possibility that an inference model suitable for the data of the inference processing target is selected.

[0255] In addition, by using machine learning, an inference model that performs the complex inference processing of selecting a second inference model from the characteristics of sensing data can be generated as a first inference model more easily than with other methods.

[0256] Furthermore, by generating data conforming to the format of the model selection information from the second inference result for training, the first inference model can be trained without being restricted by the format of the second inference result for training. In other words, various second inference models that output various second inference results can be applied to the embodiments of the present disclosure.

[0257] In addition, since the second inference model that executes a task that contributed to at least one second inference result is preferentially included in training model selection information as reference data for machine learning, it is possible to automatically generate a first inference model that outputs model selection information for selecting the second inference model that executes the contributing task. In addition, the reference data contains more data that is effective for training the first inference model, improving the efficiency of training.

[0258] In addition, since the second inference model that executes tasks belonging to a task set whose score is higher than the threshold is preferentially included in the training model selection information as reference data during machine learning, the level of precision or accuracy of the second inference result by the second inference model that is actually selected by the first inference model can be brought closer to a predetermined standard.

[0259] In addition, since the second inference model that satisfies the performance requirements is preferentially included in the training model selection information as reference data for machine learning, it is possible to automatically generate a first inference model that outputs model selection information that selects the second inference model that satisfies the performance requirements. In addition, the reference data contains more data that is effective for training the first inference model, improving the efficiency of training.

[0260] In addition, since the second inference model having performance according to the level of difficulty is preferentially included in the training model selection information used as reference data during machine learning, the processing performance (in other words, the computational cost) of the inference processing by the second inference model that is actually selected by the first inference model can be brought closer to the required level.

[0261] Furthermore, since the size information of the neural network is used as the performance of the second inference model, the performance of the second inference model can be adjusted more easily.

[0262] In the above embodiment and the above modification, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software for realizing the information processing device of the above embodiment and the above modification is a program such as the following.

[0263] In other words, this program is a program that causes a computer to execute an information processing method that is executed by a processor using a memory, in which sensing data is input into a first inference model, a first inference result for the sensing data is obtained from the first inference model, the first inference result is input into a third inference model, model selection information indicating at least one second inference model selected from a plurality of second inference models is obtained from the third inference model, the first inference result is input into the at least one second inference model indicated by the model selection information, at least one second inference result for the first inference result is obtained from the second inference model, and the at least one second inference result is output.

[0264] In addition, this program is a program that causes a computer to execute an information processing method that is executed by a processor using a memory, in which sensing data is input into a first inference model, a first inference result for the sensing data and model selection information indicating at least one second inference model selected from a plurality of second inference models are obtained from the first inference model, the first inference result is input into the at least one second inference model indicated by the model selection information, at least one second inference result for the first inference result is obtained from the second inference model, and the at least one second inference result is output.

[0265] Although the information processing device according to one or more aspects has been described based on the embodiment, the present invention is not limited to this embodiment. As long as it does not deviate from the spirit of the present invention, various modifications conceived by a person skilled in the art to this embodiment and forms constructed by combining components in different embodiments may also be included within the scope of one or more aspects.

[0266] For example, in each of the above embodiments, the inference process is described as object detection, pose estimation, and attribute determination, but is not limited to this and may be scene estimation, segmentation, estimation of distance to an object, or the like.

[0267] Also, for example, in each of the above embodiments, examples have been described in which the first information is information indicating a task, but the first information may be information indicating a second inference model. Specifically, instead of information indicating a task that contributed to at least one second inference result for training, the model selection information may include information indicating a second inference model that executes the task. Also, the model selection information may include information indicating a second inference model that output a second inference result that contributed to at least one second inference result for training. [Industrial Applicability]

[0268] The present invention can be used in a system for activity recognition. [Explanation of symbols]

[0269] 1. Training System 2. Inference Systems 10 Sensing Data 20 images 30 Model Library 40 Device Information 50 Results information 101, 201, 301, 401, 501, 601, 701, 801 Feature Extraction Unit 102, 202, 302, 402, 502, 602, 702, 802 Inference control unit 103, 303, 503, 703 Integrated Department 104, 504, 704 Task selection correct answer generation part 105, 505 Task Selection Model Training Department 208, 608 Task selection section 210, 410, 610, 810 Model Selection Section 306, 506, 706 Performance Selection Correct Answer Generation Unit 307, 507 Performance Selection Model Training Department 409, 609 Performance Selection Section 711 Training Department L1, L3, L5, M3 feature extraction models M1 Task Selection Model M2 Performance Selection Model

Claims

1. An information processing method executed by a processor using a memory, comprising: inputting sensing data into a first inference model to obtain a first inference result for the sensing data from the first inference model; inputting the first inference result into a third inference model to obtain model selection information from the third inference model, the model selection information indicating at least one second inference model selected from a plurality of second inference models; inputting the first inference result into the at least one second inference model indicated by the model selection information, thereby obtaining at least one second inference result for the first inference result from the second inference model; outputting said at least one second inference result; Information processing methods.

2. The third inference model is inputting training sensing data into the first inference model to obtain a training first inference result for the training sensing data from the first inference model; A model trained using machine learning based on at least one training second inference result obtained by inputting the training first inference result to at least one selected second inference model for each selection pattern of at least one second inference model among the plurality of second inference models, and training model selection information, which is the model selection information obtained by inputting the training first inference result to the third inference model. The information processing method according to claim 1 .

3. Training the third inference model includes training using machine learning in which data generated in accordance with the format of the model selection information based on the at least one training second inference result is used as reference data, and the training model selection information is used as output data. The information processing method according to claim 2 .

4. The model selection information includes first information corresponding to a task to be performed by a second inference model; The at least one second inference model indicated by the model selection information executes a task corresponding to the first information; and The reference data is generated such that a second inference model that executes a task that contributed to the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information. The information processing method according to claim 3.

5. The generation of the reference data includes: generating the reference data such that a second inference model that executes a task belonging to a task set in which a score based on the at least one training second inference result is higher than a threshold is preferentially included in the at least one second inference model indicated by the training model selection information. The information processing method according to claim 4.

6. the model selection information includes second information corresponding to a performance of the second inference model; At least one second inference model indicated by the model selection information has a performance corresponding to the second information; The generation of the reference data includes: generating the reference data such that a second inference model that satisfies a performance requirement in the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information. The information processing method according to any one of claims 3 to 5.

7. The second information includes a degree of difficulty in processing the second inference model; Reference data generation is generating the reference data such that a second inference model having a performance according to the difficulty level is preferentially included in at least one second inference model indicated by the training model selection information; The information processing method according to claim 6.

8. the second inference model includes a neural network model; The second information includes size information indicating a size of the neural network model as a performance of the second inference model, Reference data generation is generating the reference data such that a second inference model having the size information included in the second information is preferentially included in at least one second inference model indicated by the training model selection information. The information processing method according to claim 6.

9. An information processing method executed by a processor using a memory, comprising: inputting sensing data into a first inference model, and acquiring from the first inference model a first inference result for the sensing data and model selection information indicating at least one second inference model selected from a plurality of second inference models; inputting the first inference result into the at least one second inference model indicated by the model selection information, thereby obtaining at least one second inference result for the first inference result from the second inference model; outputting said at least one second inference result; Information processing methods.

10. The first inference model is inputting training sensing data into the first inference model to obtain a training first inference result for the training sensing data and training model selection information from the first inference model; A model trained by machine learning based on at least one training second inference result obtained by inputting the training first inference result into at least one selected second inference model for each selection pattern of at least one second inference model among the plurality of second inference models, and on the training model selection information. The information processing method according to claim 9.

11. Training the first inference model includes training using machine learning in which data generated in accordance with the format of the model selection information based on the at least one training second inference result is used as reference data, and the training model selection information is used as output data. The information processing method according to claim 10.

12. The model selection information includes first information corresponding to a task to be performed by a second inference model; The at least one second inference model indicated by the model selection information executes a task corresponding to the first information; and The reference data is generated such that a second inference model that executes a task that contributed to the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information. The information processing method according to claim 11.

13. The generation of the reference data includes: generating the reference data such that a second inference model that executes a task belonging to a task set in which a score based on the at least one training second inference result is higher than a threshold is preferentially included in the at least one second inference model indicated by the training model selection information. The information processing method according to claim 12.

14. the model selection information includes second information corresponding to a performance of the second inference model; At least one second inference model indicated by the model selection information has a performance corresponding to the second information; The generation of the reference data includes: generating the reference data such that a second inference model that satisfies a performance requirement in the at least one training second inference result is preferentially included in the at least one second inference model indicated by the training model selection information. The information processing method according to any one of claims 11 to 13.

15. The second information includes a degree of difficulty in processing the second inference model; Reference data generation is generating the reference data such that a second inference model having a performance according to the difficulty level is preferentially included in at least one second inference model indicated by the training model selection information; The information processing method according to claim 14.

16. the second inference model includes a neural network model; The second information includes size information indicating a size of the neural network model as a performance of the second inference model, Reference data generation is generating the reference data such that a second inference model having the size information included in the second information is preferentially included in at least one second inference model indicated by the training model selection information. The information processing method according to claim 14.

17. a first acquisition unit that inputs sensing data into a first inference model to acquire a first inference result for the sensing data from the first inference model; a second acquisition unit that acquires model selection information indicating at least one second inference model selected from a plurality of second inference models from a third inference model by inputting the first inference result into the third inference model; a third acquisition unit that acquires at least one second inference result for the first inference result from the second inference model by inputting the first inference result to the at least one second inference model indicated by the model selection information; and an output unit that outputs the at least one second inference result. Information processing system.

18. a first acquisition unit that inputs sensing data into a first inference model to acquire from the first inference model a first inference result for the sensing data and model selection information indicating at least one second inference model selected from a plurality of second inference models; a second acquisition unit that acquires at least one second inference result for the first inference result from the second inference model by inputting the first inference result to the at least one second inference model indicated by the model selection information; and an output unit that outputs the at least one second inference result. Information processing system.

Citation Information

Patent Citations

  • Large scale neural network constructing method and device and neural network device

    JP1998063632A

  • Program, device, and method for estimating context using a plurality of recognition engines

    JP2019139479A

  • Server device, trained model providing program, trained model providing method, and trained model providing system

    WO2018173121A1