Information processing apparatus, information processing method, and storage medium

Abstract

The present application provides an information processing device, an information processing method, and a storage medium. The information processing device includes a first acquisition unit configured to acquire information about learning data used in learning of each of a plurality of pre-learning models for identifying input data, a second acquisition unit configured to acquire information indicating an attribute of identification target data, and a model selection unit configured to select, from the plurality of models, a model to be used in identification of the identification target data, based on a degree of matching between the attribute of the identification target data and attributes of the learning data used in learning of each of the plurality of models, and based on diversity of the attributes of the learning data used in learning of each of the plurality of models.

Description

TECHNICAL FIELD

[0001] The present application relates to an information processing apparatus, an information processing method, and a storage medium, and more particularly, to a technology for selecting a learning model. BACKGROUND

[0002] In machine learning, a learning model is formed by learning of learning data. If the data contents of learning data sets are different, models having different characteristics are formed. When a plurality of models are learned using a plurality of different learning data sets as described above, an appropriate model must be selected in accordance with an identification target.

[0003] PTL 1 discloses a technology by which a model learned using an image having imaging conditions similar to imaging conditions including an imaging position and an imaging angle of an identification target image is selected as a model to be used in identification.

[0004] Further, a model learned by using a learning data set formed of various data is generally able to cope with various identification target data. On the other hand, a model learned by using a learning data set formed of limited data can cope only with identification target data that changes little. However, this model can identify identification target data having conditions included in the learning data set with higher performance than that of a model learned using various data. Therefore, if there are a plurality of models obtained by different learning data sets, it is desirable to select a model learned by using a learning data set formed of limited data as much as possible that matches the properties of the identification target data.

[0005] LIST OF CITATIONS

[0006] PATENT LITERATURE

[0007] Patent Literature 1: Japanese Patent No. 6474946 SUMMARY

[0008] PROBLEMS

[0009] Unfortunately, the technology described in PTL 1 can select only a model having similar imaging conditions. This causes a problem in that it is difficult to select a model learned by using data corresponding to the properties of an identification target and having high identification performance with respect to the identification target from a plurality of models.

[0010] The present application is made in view of the above-described problem, and provides a technology for selecting a model learned by using data corresponding to the properties of an identification target and having high identification performance with respect to the identification target from a plurality of models.

[0011] SOLUTION TO THE PROBLEM

[0012] An information processing apparatus according to an aspect of the present invention is provided to achieve the above object, and includes a first acquisition unit configured to acquire information on learning data used in learning of each of a plurality of pre-learned models for identifying input data, a second acquisition unit configured to acquire information indicating an attribute of identification target data, and a model selection unit configured to select, from the plurality of models, a model to be used in identification of the identification target data, based on a degree of matching between the attribute of the identification target data and an attribute of the learning data used in learning of each of the plurality of models, and based on diversity of the attribute of the learning data used in learning of each of the plurality of models.

[0013] Advantages of the Invention

[0014] According to the present invention, it is possible to select, from a plurality of models, a model that has learned by using data corresponding to an attribute of an identification target and having high identification performance with respect to the identification target.

[0015] Other features and advantages of the present invention will become apparent from the following description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention. It is to be expressly understood, however, that the drawings are provided for the purpose of illustration only and are not intended to limit the scope of the invention. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0017] Figure 1 is a diagram illustrating an example of a hardware configuration of an information processing apparatus according to a first embodiment;

[0018] Figure 2 is a block diagram illustrating an example of a functional configuration of an information processing apparatus according to the first embodiment;

[0019] Figure 3 is a diagram for explaining learning data and a learning model according to the first embodiment;

[0020] Figure 4 is a flowchart illustrating a process by the information processing apparatus according to the first embodiment;

[0021] Figure 5 is a diagram illustrating a user interface for setting an attribute of identification target data according to the first embodiment;

[0022] Figure 6 is a flowchart illustrating a process to be performed by the model selection unit according to the first embodiment;

[0023] Figure 7This is a diagram used to explain the model selection process according to the first embodiment;

[0024] Figure 8 It is a graph used to interpret the learning data and target identification data according to the third embodiment;

[0025] Figure 9 This is a diagram used to explain the user interface according to the fourth embodiment;

[0026] Figure 10 This is a diagram illustrating an example of the usage of the information processing apparatus according to the fourth embodiment;

[0027] Figure 11 This is a block diagram illustrating an example of the functional structure of an information processing apparatus according to the fifth embodiment;

[0028] Figure 12 This is a diagram illustrating a user interface for inputting user evaluation according to a fifth embodiment; and

[0029] Figure 13 This is a diagram illustrating a user interface for setting attributes of target data for identification according to a sixth embodiment. Detailed Implementation

[0030] In the following description, embodiments will be illustrated in detail with reference to the accompanying drawings. Please note that the following embodiments are not intended to limit the scope of the claimed invention. Several features are described in the embodiments, but the invention is not limited to requiring all of these features, and multiple such features can be suitably combined. Furthermore, in the drawings, the same reference numerals are given the same or similar constructions, and their redundant descriptions are omitted.

[0031] (First Embodiment)

[0032] Using a model that has already been learned using a training dataset with attributes that perfectly match those of the target data is advantageous for identifying the target data. If such a learning model exists, it can simply be selected as the suitable model for identification. However, if the attribute types are diverse, it becomes difficult to prepare a model that has already been learned using a training dataset with attribute patterns corresponding to all combinations. For example, with 10 attributes relative to the data, if the dataset consists of combinations of "included" and "excluding" for each attribute, a dataset with 1023 patterns is formed. In this case, if the model is learned for each dataset, 1023 learning iterations must be performed. If the number of attribute types is increased by subdividing the data conditions, the number of models to be learned increases further. Since learning typically requires significant computational time, it is difficult to form a learning model with all attribute patterns.

[0033] Therefore, in reality, it is possible to prepare a model with attribute patterns that include only some of the possible combinations of all attribute patterns. This makes it difficult to always select a model with attribute patterns that perfectly match the attribute patterns of the target data being identified.

[0034] On the other hand, when using a model that has not learned the attributes included in the target data for recognition, the target data may not be adequately recognized. This necessitates selecting a model that has already learned the attributes of the target data. However, a simple and reliable method for selecting a model that has learned all attributes is to choose a model that has learned all attributes. However, if the attributes of the target data are limited, recognition performance relative to the target data improves when selecting a model that includes the attributes of the target data but has learned data with as few additional attributes as possible, compared to selecting a model that has learned all attributes.

[0035] In the first embodiment, the method of selecting a model that includes the attributes of the target data but has been learned with as few additional attributes as possible will be explained by taking the learning of a defect identification model in infrastructure inspection and the selection of a model suitable for the structural image of the identified target as an example.

[0036] First, the inspection of infrastructure will be explained. In the inspection of infrastructure walls, inspectors record defects such as cracks in concrete walls. In image-based inspections, inspectors take images of the structure's walls and record the location and extent of defects based on the images, thus forming the inspection results. The resulting inspection results and images are then managed in conjunction with the structure's drawings. In this case, finding and recording all defects from the images is a very challenging task. Therefore, recent attempts have focused on improving the efficiency of image inspection by using a learning recognition model to identify defects in concrete wall images through machine learning.

[0037] In this embodiment, the identification model used to identify defects is an image recognition model for identifying cracks from concrete wall images. Note that the defect identification model for infrastructure inspection is not limited to crack identification models, but can also be a model for identifying other defects. For example, the model could be an identification model for identifying leaking areas, peeling areas, or rust areas, and could also be a model for determining the degree of deterioration of a given concrete wall area through machine learning.

[0038] In this embodiment, the data has the following characteristics, which will be described in detail later. First, a model for identifying cracks is learned using learning data and training data. The learning data includes images of concrete walls containing cracks, while the training data indicates the correct crack locations in the images. Furthermore, an example of the data attribute in this embodiment is the type of infrastructure, such as a bridge or tunnel. A model that has learned this attribute using images of all types of structures is applicable to any structure type. However, if the target image is known to be a bridge, high-precision results can be obtained by using a model learned only using bridge images (data with only the structure type attribute set to "bridge").

[0039] Reference Figure 5 The attributes of this embodiment are further explained. These attributes indicate the conditions of data contained in the learning dataset or the identification target data. These attributes include various items, and each item will be referred to as an attribute item below. An example of an attribute item is the structure type of infrastructure. Examples of structure types are "bridge," "tunnel," and "dam." That is, one example of an attribute item is the type of identification target indicated by the data. Another example of an attribute item is the wall condition of the infrastructure. Figure 5 Examples of wall conditions, such as "numerous cracks" and "numerous rust," are shown. That is, attribute items can include the appearance of the image data and the pattern of wall deterioration. Figure 5 Camera parameters are further illustrated as examples of attribute items based on imaging conditions. As examples of camera parameters, Figure 5 The exposure parameters "below EV-2" and "above EV+2" are shown. Imaging conditions can include not only the imaging parameters described above, but also information such as camera type and the weather and season at the time of imaging. That is, the attributes in this embodiment can include all items used for classifying the data. For convenience, Figure 5 Only the attribute items mentioned above are shown, but more attribute items can be added and their content further subdivided. Please note that... Figure 5 The rest will be explained later.

[0040] The following will refer to Figure 1 and Figure 2 The structure of the information processing apparatus 100 according to this embodiment is explained.

[0041] <Hardware Structure of Information Processing Devices>

[0042] Figure 1 This is a diagram illustrating the hardware structure of the information processing apparatus 100 according to this embodiment. Figure 1As shown, the information processing device 100 includes a CPU 101, a ROM 102, a RAM 103, an HDD 104, a display unit 105, an operation unit 106, and a communication unit 107. The CPU 101 is the central processing unit, performing arithmetic operations, logical determinations, etc., for various processes, and controlling the components connected to the system bus 108. The ROM (Read-Only Memory) 102 is the program memory and stores programs controlled by the CPU 101, including the processes described later. The RAM (Random Access Memory) 103 serves as a temporary storage area, such as a working area or the main memory of the CPU 101. Note that the program memory can also be implemented by loading programs from external storage devices connected to the information processing device 100 into the RAM 103.

[0043] HDD 104 is a hard disk for storing electronic data and programs according to this embodiment. An external storage device can also be used as a similar device. This external storage device can be implemented using a medium (recording medium) and an external storage drive for accessing that medium. Known examples of such media are floppy disks (FD), CD-ROMs, DVDs, USB storage devices, MO drives, and flash memory. The external storage device can also be a server device connected via a network.

[0044] Display unit 105 is, for example, a CRT monitor or an LCD monitor, and is a device for outputting images to a display screen. Note that display unit 105 can also be an external device connected to information processing device 100 via a wired or wireless connection. Operation unit 106 includes a keyboard, mouse, etc., and receives various operations from the user. Communication unit 107 performs wired or wireless bidirectional communication with, for example, other information processing devices, communication devices, and external storage devices using known communication technologies.

[0045] Functional Structure of Information Processing Devices

[0046] Figure 2 This is a block diagram example illustrating the functional configuration of the information processing apparatus 100 according to this embodiment. The information processing apparatus 100 includes a learning unit 201, a model storage unit 202, an attribute processing unit 203, a model selection unit 204, and a recognition unit 205. These functional units are implemented by the CPU 101 by loading a program stored in the ROM 102 into the RAM 103 and executing processes conforming to the respective flowcharts (described later). The CPU 101 stores the execution results of each process in the RAM 103 or the HDD 104. For example, when constructing hardware instead of using software processing with the CPU 101, it is only necessary to form circuits or arithmetic units corresponding to the processing of the functional units to be explained herein.

[0047] The following will explain Figure 2 The following is a summary of the functional units shown. The learning unit 201 learns multiple models. The model storage unit 202 stores the multiple learned models in a recording device such as HDD 104. The attribute processing unit 203 processes attribute information of the target data and transmits the processed information to the model selection unit 204. The model selection unit 204 receives attribute information of the target data from the attribute processing unit 203 and obtains information about the learned models from the model storage unit 202. Based on this information, the model selection unit 204 selects a model suitable for recognizing the target data. The recognition unit 205 performs recognition processing on the target data using the model selected by the model selection unit 204. Note that not all functional units need to be included in the information processing device 100; rather, at least some functions can be implemented by external devices that can be connected to the information processing device 100. For example, the functions of the learning unit 201 and the recognition unit 205 can be implemented by external devices, and the model storage unit 202 can be installed in an external device. The required information can also be exchanged by appropriate communication with external devices, thereby minimizing the processing of the information processing device 100.

[0048] In this embodiment, the learning unit 201 pre-learns multiple models and stores them in the model storage unit 202. The learning of multiple models and the properties of the learning datasets for each model will be explained below.

[0049] First, in this embodiment, as described above, the learning data for learning a model to identify cracks in images comprises pairs of images of a concrete wall and training data indicating the location of cracks in the images. A learning dataset is obtained by collecting a large number of image-training data pairs. The model is then learned using this learning dataset. In this embodiment, model learning can be performed using any machine learning algorithm (e.g., an algorithm such as a neural network).

[0050] In this embodiment, training data with various conditions is prepared to identify images under various imaging conditions and with various infrastructures. In the following explanation, all the prepared training data will be referred to as the total training data. The model learned using this total training data as the training dataset is a robust model that can be used under various conditions.

[0051] On the other hand, in order to form a model specifically designed for finite conditions, learning data specific to these finite conditions is extracted from the total learning data, forming a learning dataset that is a subset of the total learning data. Compared to a model that has already been learned using the total learning data, a model learned using this subset of the learning dataset can achieve high-performance results under finite conditions. Furthermore, the attribute set of the learning dataset is recorded in association with the learning model.

[0052] The following will refer to Figure 3 Explain the model learning and the attribute set of the learning dataset performed by the learning unit 201 according to this embodiment. Figure 3 This illustrates how the learning datasets D1 to D3 are formed into subsets of the total learning data, and how models M1 to M3 are formed by learning from each of the learning datasets.

[0053] Balloon 300 indicates the data included in the total learning data, and includes a large amount of learning data, each of which includes a pair of training data 311 indicating the location of cracks in the concrete wall image 310. Furthermore, information 312 indicating the attributes of each learning data point is associated with the learning data.

[0054] Balloon 301 indicates an example of information indicating an attribute of given learning data. Balloon 301 includes, for example, the attribute item "bridge," and displays information indicating whether the learning data image is an image of a bridge. In this embodiment, the attribute item is set by two values ​​(i.e., "true" and "false"). In the example of balloon 301, the learning data image is a bridge image, therefore "true" is set in the attribute item "bridge."

[0055] To create learning data suitable for a model based on predetermined conditions, a learning dataset is formed by extracting learning data based on predetermined conditions from the total learning data. For this purpose, attributes indicating predetermined conditions are set, and the learning dataset is formed by collecting learning data from the total learning data that matches these attributes. For example, to form a recognition model suitable for bridges, a learning dataset can be formed by extracting only bridge images from the total learning data. Alternatively, the conditions can be further narrowed, and images of bridges and walls with many cracks can be extracted to form a learning dataset. Comparatively, a learning dataset including both bridges and tunnels can also be formed, thus creating a learning dataset for learning a relatively robust model. As a method for forming learning datasets as described above, learning data can be manually distributed and formed. Alternatively, learning datasets containing various attribute combinations can be automatically formed based on information about the attributes associated with each piece of learning data.

[0056] Then, the attributes of the learning data that form a subset of the total learning data are summed. Figure 3Balloon 302 in the diagram indicates the sum of the attributes of the learning data included in the learning dataset D3. The learning dataset D3 includes not only learning data for bridge images but also learning data for tunnel images. Therefore, for the attributes in balloon 302, "true" is set in the attribute items "bridge" and "tunnel". The information obtained as described above by summing the attributes of the data included in the learning dataset for learning and recording the "true" or "false" of each attribute item will be referred to hereinafter as the attribute set. As mentioned above, the learning data contains multiple data points for learning, and the attributes of the learning data are determined based on the sum of the attributes of these multiple data points.

[0057] As a method for forming learning datasets, multiple learning datasets can be randomly generated based on the total learning data, and the attributes of the learning data included in each learning dataset can be collected to form an attribute set of the learning dataset.

[0058] As described above, the attribute set of each learning dataset is formed based on the attributes of the learning data contained in the learning dataset. The information of the attribute set is stored in the model storage unit 202 along with the model learned using each learning dataset. This allows the model selection unit 204 (described later) to reference the contents of the learning datasets on which the model has already been learned by invoking the attribute set along with the model.

[0059] Please note that the above explanation of the total learning data already included an example of how the information indicating the attribute is associated with each piece of learning data in a one-to-one correspondence. However, the invention is not limited to this; information indicating attributes in the total learning data can also be prepared for multiple pieces of learning data together. For example, information setting "true" in the attribute item "bridge" can be assigned all at once to multiple pieces of learning data containing bridge wall images.

[0060] In the embodiments explained above, attribute information is pre-assigned to each piece of learning data in the total learning data, and the attribute set of the learning dataset is formed by adding the attributes of the learning data contained in the learning dataset.

[0061] However, the method for forming the attribute set of the learning dataset is not limited to this; another method can also be used. For example, the attribute set of a given learning dataset can be formed by having a human examine the image data and attribute information of the learning data included in the learning dataset, and by having a human set the attributes of the learning dataset to "true" or "false". Furthermore, when the learning dataset contains only a small amount of learning data with a given attribute, it can be determined that the learning effect of that attribute is small, and even if the learning dataset contains that attribute, the attribute will be set to false. Suppose that in order to learn a model for recognizing tunnel images, a learning dataset is formed by primarily collecting tunnel images, and this learning dataset also includes learning data for several bridge images. In such cases, "true" can be set as an attribute of the tunnel, and "false" can be set as an attribute of the bridge.

[0062] As described above, model storage unit 202 stores the learned model and the attribute set of the learning dataset that learned the model. For simplicity, in some cases below, the attributes of the learning dataset that learned the model will be referred to as the attribute set of the model.

[0063] <Processing>

[0064] The following will refer to Figure 4 The flowchart shown explains the overall processing procedure to be performed by the information processing device 100 according to this embodiment.

[0065] In step S401, the attribute processing unit 203 forms information about the attribute set of the target data. Before explaining step S401 in detail, the target data for identification in this embodiment will be explained first.

[0066] The target data for identification in this embodiment consists of images of a specific infrastructure that is the target of inspection. As inspection images for infrastructure, images with very high resolution are captured to inspect for defects such as cracks in concrete walls. Therefore, in order to record all the walls of the infrastructure in image form, many images must be captured. That is, to inspect a given infrastructure, the process of identifying cracks from the images is performed on a large number of images. Thus, the target data for identification in this embodiment contains a large number of images.

[0067] Since a large number of images are formed from a single infrastructure, the diversity of these images is lower than that of images from all infrastructures. To examine a given infrastructure, recognition processing is performed on these images with limited conditions, thus achieving good recognition performance when selecting a model that matches the target data.

[0068] In order to select a model that matches the target data for identification, the attribute processing unit 203 performs processing to form an attribute set for the target data for identification in step S401. Similar to the attribute set of the learning dataset, the attribute set of the target data for identification consists of information that sets "true" or "false" for each attribute item.

[0069] An example of a method for setting the attribute set of target data is a method where a person checks the content of the target data and inputs information about each attribute. The above... Figure 5 An example of a user interface for setting up the target data for identification is shown. As described later, the model selection unit 204 selects a model based on the content of the attribute set of the target data and the attribute set of the learning dataset. Therefore, the target data for identification has the same attribute items as the attribute items in the learning dataset. More specifically, in Figure 3 The items in the attribute set of balloon 302 and Figure 5 The items in the attribute set shown are the same.

[0070] As mentioned above, the target data for identification is formed from multiple images. Therefore, the target data for identification contains images with various attributes. For example, the attributes for structural types are as follows. When the target data for identification is an image of a bridge, "True" is set in the attribute item "Bridge," and "False" is set in the attributes for other structural types.

[0071] The following explanation uses camera parameter attributes as an example to illustrate the attributes of target recognition data. Assume that when imaging a wall of a structure, many structural parts are imaged at an exposure of EV0. However, if there are dark areas of the structure, such as shadows, only that part is imaged at an exposure of EV+2 in some cases. In contrast, a wall illuminated by strong sunlight might sometimes be imaged at an exposure of EV-2 by reducing the exposure. As attributes of target recognition data formed from images taken as described above, both the exposure EV-2 and exposure EV+2 attributes are set to "True".

[0072] The user interface for setting attributes of the target data to be identified will be explained below. The user is someone who operates the information processing device 100 of this embodiment and wants to perform the process of selecting a learning model suitable for the target data to be identified. In addition, the attribute processing unit 203 performs processing related to the following user interface. The attribute processing unit 203 performs, for example, the following processes: processing of displaying GUI information on the display unit 105, receiving information set by the user via the operation unit 106, and recording the set information in the recording area.

[0073] exist Figure 5 In the user interface shown, the user examines the content of the target data and sets various attributes. Figure 5The user interface shown illustrates a GUI example for setting "True" and "False" via radio buttons for each attribute item. Note that this is not the only way to implement a user interface; other interfaces can also be used. For example, attributes can be set using dropdown menus. Figure 5 In the example shown, the structure for identifying the target data is a bridge. Therefore, Figure 5 This demonstrates the state of "Bridge" set to "True" and "Tunnel" set to "False" using mouse cursor 501. The user can then... Figure 5 Use the status shown to set other attribute items.

[0074] The initial value can also be set to an attribute that identifies the target data. (See reference...) Figure 5 "Void" is set to an attribute that the user has not set. "Void" indicates that the attribute is unknown. "Void" is set as the initial value when the user cannot determine a given attribute of the target data and has not entered anything for that attribute. When an attribute setting includes "Void," the attribute item with "Void" set is not used for model selection in the processing performed by the model selection unit 204 (described later). An example of setting "Void" as the initial value of an attribute has been explained above, but it is also possible to set "False" instead of "Void" as the initial value.

[0075] When the attributes are set as described above, information is generated indicating whether each attribute of the target data is "true" or "false". This information is used as the attribute set for identifying the target data. Note that patterns with the attribute set to "invalid" can also be included when "invalid" information is also used.

[0076] The above has explained an embodiment in which a user checks the content of the target data and inputs the attributes of the target data, but the present invention is not limited thereto. For example, the attributes of the target data can also be automatically determined and set. In this embodiment, the attribute processing unit 203 performs processing to determine the content of the target data and sets the attributes of the target data. When determining the content of the target data, processing is performed to determine the content of each attribute item. For example, the attribute items of camera parameters are processed as follows: that is, camera parameters such as exposure settings and ISO sensitivity are obtained by reading the imaging information of the image attached to the target data. Based on the obtained camera parameters, each attribute item related to the camera parameters is set to "true" or "false". In addition, when imaging location information such as latitude and longitude of the captured image can be obtained, the type of structure in the imaging location can be determined based on the location information and a database of structures. Attribute items that cannot be determined by the information attached to the image can be processed as follows: that is, an image discriminator is prepared for determining each attribute, and the attribute is set to "true" or "false" based on the discrimination result of the image discriminator. For example, an image discriminator that has been trained to distinguish between images containing many cracks and images containing few cracks can be used to determine whether the target data contains an image with many cracks and to set attributes related to the number of cracks. Furthermore, it is not necessary to automatically determine all attributes. That is, attribute determination results can be set for attributes that can be automatically determined, and the user can be prompted to input attributes that cannot be automatically determined.

[0077] As described above, the information for identifying the attribute set of the target data is formed through the processing of the attribute processing unit 203.

[0078] [S402]

[0079] Then, in step S402, the model selection unit 204 selects a model suitable for recognizing the target data from multiple models based on the attribute set of the learning dataset and the attribute set of the target data. More specifically, the model selection unit 204 selects a model by progressively determining the matching degree between the attribute set of the learning dataset and the attribute set of the target data, as well as the diversity of the learning dataset.

[0080] Figure 7 This is a diagram illustrating an example of the attribute set for identifying target data and the attribute sets for multiple (n) models. For convenience, Figure 7 The symbols ○ and × represent the "true" and "false" attributes. Figure 7Models M1 through M4 shown are models learned using a training dataset that is a subset of the total training data, and models learned using training data that includes a limited set of attributes. For example, when focusing on the attribute of structure type, models M1 through M3 are models learned using training data of bridge images, and model M4 is a model learned using training data of tunnel images. Furthermore, model Mn is a model learned using the total training data, and is a model where all attribute items are "true".

[0081] The following will refer to Figure 6 and Figure 7 The flowchart shown explains in detail the processing of model selection unit 204. Figure 6 The flowchart illustrates the detailed process of the processing in step S402 performed by the model selection unit 204.

[0082] In step S601, the model selection unit 204 determines the matching degree between the attribute set of the learning dataset and the attribute set of the target data. In this embodiment, the matching degree between the attribute sets is determined by examining the inclusion relationship between the attribute set of the target data and the attribute set of the learning dataset. More specifically, by focusing on the attribute items set to "true" in the target data, models that set the same attribute items to "true" are selected. Figure 7 In the example shown, the attribute items "Bridge", "Many Cracks", and "Below EV-2" for identifying the target data are set to "True".

[0083] The following model is originally the most suitable model for identifying the target data. In this model, the three types of attribute items are set to "true" and the model has a perfectly matching attribute set. In step S601, the model selection unit 204 checks whether a model with a perfectly matching attribute set exists. If a model with a perfectly matching attribute set exists, the model selection unit 204 selects that model. If a model with a perfectly matching attribute set exists, the process of determining diversity in step S602 is unnecessary. Therefore, the process can be terminated by skipping step S602.

[0084] However, as mentioned earlier, it is difficult to learn multiple models that include patterns across all attribute sets, thus creating situations where the best model must be selected from models with imperfectly matching attribute sets. Figure 7 In the example shown, there is no model with an attribute set that perfectly matches the attribute set of the target data.

[0085] In situations like this, model selection unit 204 selects a model that includes the attributes of the target data in step S601. More specifically, model selection unit 204 selects models that set the attribute items "bridge," "many cracks," and "below EV-2" to "true" in the same manner as in the target data. This is because, to address the attributes of the target data, it is necessary to select a model that has learned the conditions for these attributes. Based on this criterion, in Figure 7 The example shown selects three models: M2, M3, and Mn. If multiple models are therefore selected in step S601, the models are further subject to the processing in step S602.

[0086] In step S602, the model selection unit 204 selects a model based on the diversity of the attribute set of the models selected in step S601. To perform recognition with high performance, it is desirable to select a model that has been learned from the attributes of the target data, including any additional learning data. Therefore, in step S602, models that have been learned using learning data with low (small) attribute diversity are selected from the models selected in step S601, thus selecting models that include the attributes of the target data and have not undergone any additional learning. Model diversity is calculated by counting the number of attributes in the learning dataset of the learned models. More specifically, the number of attributes set to "true" in the learning dataset of each model is calculated as diversity. Figure 7 In the example shown, the diversity of the models selected in step S601 is 4 for model M2, 5 for model M3, and 7 for model Mn. In this diversity-based model selection in step S602, the model with the lowest diversity value is selected from the models chosen in step S601. Therefore, in Figure 6 In the example shown, model M2 is selected.

[0087] According to Figure 4 The flowchart explains the details of the process for selecting a suitable model for recognizing the target data. Note that if no model containing the attributes of the target data is available in step S601, the model with the highest attribute matching degree (the model containing the most attributes of the target data) is selected. In this case, the attribute items that must be matched can also be preset. For example, structure type is an important attribute item in model selection. Therefore, it is also possible to select a model with at least structure type attribute settings that match, and select the model with the best matching for other attribute items from the models selected based on structure type.

[0088] As mentioned earlier, for the attributes of the target data, "invalid" can be set as information indicating that the attribute is unknown. In this case, the model selection unit 204 can perform processing by ignoring attributes set to "invalid" in the attribute items of the target data. More specifically, in the process of determining the matching degree between the attributes of the target data and the attributes of the model, it is only necessary to prevent the attribute items set to "invalid" in the attribute set of the target data from affecting the matching degree of the model, without considering whether the attribute items of the model are "true" or "false".

[0089] As described above, the model selection unit 204 selects a model suitable for recognizing the target data.

[0090] [S403]

[0091] In step S403, the recognition unit 205 performs recognition processing using the model selected in step S402 and forms a recognition result relative to the target image. In this embodiment, the recognition unit 205 identifies cracks in the target image and forms a recognition result indicating the location of the cracks in the image. This completes the process. Figure 4 The series of processes shown.

[0092] As described above, this embodiment obtains information about the learning data used in the learning of each of the multiple models to be used for recognizing input data, as well as information indicating the attributes of the target data to be recognized. Then, based on the matching degree between the attributes of the target data and the attributes of the learning data used in the learning of each of the multiple models, and the diversity of the attributes of the learning data used in the learning of each of the multiple models, a model for recognizing the target data is selected from the multiple models. That is, the model selected in this embodiment is a model that includes the attributes of the target data and has not learned any additional attributes. Therefore, a model that has been learned by using data corresponding to the attributes of the target data and has high recognition performance relative to the target can be selected from the multiple models, and thus good recognition results can be expected.

[0093] (Second Embodiment)

[0094] In the first embodiment, an example of selecting a model suitable for recognizing the target data has been explained. However, model selection is not limited to selecting one model; multiple models can also be selected based on diversity. In the second embodiment, an example of selecting multiple models suitable for recognizing the target data will be explained.

[0095] Please note that the hardware and functional configuration of the information processing apparatus 100 according to the second embodiment are the same as those of the first embodiment, and therefore their explanation will be omitted. Furthermore, as with the first embodiment, an embodiment using a crack identification model for inspecting infrastructure will also be explained in the second embodiment. The processing procedure according to the second embodiment conforms to… Figure 6 and Figure 8 The flowchart shown.

[0096] In the first embodiment, the selected model includes the attributes of the target data and has not learned any additional attributes, thus it is expected to perform target data recognition with high performance. To achieve this, the attributes of the target data need to be set correctly. However, when the amount of image data in the target data is very large, it is sometimes difficult to correctly determine all the attributes of the target data. Especially when the user sets the attributes of the target data, it is difficult to check all the target images, and this may make it impossible to correctly set attributes related to the image appearance, such as the state of the wall.

[0097] To address situations like this, in addition to the model selected in the first embodiment, the second embodiment also selects a model with high (large) attribute set diversity and robust recognition capabilities. More specifically, in step S602 explained in the first embodiment, the model with the lowest diversity is selected as the first model in the same manner as in the first embodiment. Then, a model with a relatively high diversity value is selected as the second model for robust recognition. More specifically, the model with more "true" values ​​in the attribute items than the first model is selected as the second model. That is, a model that has been learned using training data with high diversity is selected. Therefore, even if the attribute settings of the target data are incorrect, a relatively robust model is selected as the second model. Therefore, even if the diversity of the target data is higher than expected, a model that can handle the target data is selected.

[0098] If a model with the greatest diversity is selected when one has already been learned using the total learning data, then the model learned using the total learning data is always chosen. While this is one example, it is also possible to select a model whose diversity is greater than the first model by a predetermined value. This allows for the selection of a model that includes attributes of the target data and has slightly higher robustness. This model represents an intermediate performance between the total learning data model and the first model. Although this model is less robust than the model with the total learning data, it is suitable for identifying the target data due to the relatively small amount of additional learning data.

[0099] As mentioned earlier, there is a possibility that the attributes of the identification target data regarding the wall condition are not set correctly. Therefore, a model that is robust to attributes related to the wall condition can be selected as the second model. Alternatively, a model can be selected where the number of "true" values ​​set in the attribute items related to the wall condition is a predetermined number greater than the number of "true" values ​​set for the attributes of the first model. As stated above, even when setting the second model based on diversity, the second model can be selected by restricting the attributes used to evaluate diversity.

[0100] Through the above processing, based on the attribute set of the target data, a first model considered optimal for the target and a relatively robust second model can be selected.

[0101] Then, in step S403 as explained in the first embodiment, the recognition unit 205 forms recognition results for each of the two selected models and presents these two results to the user. The user checks the two recognition results and adopts the recognition result that is more suitable.

[0102] For this process, the recognition unit 205 performs the process of displaying the recognition results on the display unit 105, and the process of accepting the user's selection of the recognition results via the operation unit 106. Presenting these two recognition results allows for comparison of the recognition results of each model. More specifically, an image is formed by overlaying the crack recognition results of each model onto the target image, and the formed images are displayed side-by-side or one at a time.

[0103] The display unit 105 can also display information about the two models selected by the model selection unit 204 before the recognition unit 205 performs the recognition process, thereby allowing the user to determine the model to be used in the recognition process. The model information to be displayed here is, for example, information about the attribute sets of each model. Furthermore, to make it easier to compare the attribute set information of the models, the information can be displayed side-by-side or attribute items with differences can be highlighted. By allowing the user to pre-select models, the number of models that the recognition unit 205 processes for recognition can be reduced, and therefore the recognition processing time can also be reduced.

[0104] Please note that this embodiment has explained the example of selecting two models. However, the number of models to be selected is not limited to two; three or more models can also be selected based on diversity. In this case, multiple second models are selected based on diversity. More specifically, models with slightly higher diversity than the first model can be selected, and models with even higher diversity can be chosen. This allows for the selection of multiple models with varying robustness.

[0105] (Third Embodiment)

[0106] In the above embodiments, attributes are clear items such as structural types that humans can visually understand. However, the invention is not limited thereto. In the third embodiment, an example of forming attribute information based on image feature quantities will be explained. Note that the hardware and functional configuration of the information processing apparatus 100 according to the third embodiment are the same as those of the first embodiment, and therefore its explanation will be omitted. Furthermore, as with the first embodiment, an embodiment using a crack identification model for inspecting infrastructure will also be explained in the third embodiment.

[0107] In the third embodiment, the information of the attribute set is formed from images of the learning dataset or images of the target recognition data using a method called Bag-of-Features or Bag-of-Visual Words. Since the Bag-of-Features method is well-known, a detailed explanation of the algorithm itself will be omitted. The following will refer to... Figure 8 Explain an example of forming an attribute set using a feature bag.

[0108] Figure 8 Learning datasets 801 to 80n are shown. The process of forming attribute set 841 from learning dataset 801 in these learning datasets will be explained. To explain this process, the visual word in this embodiment will be explained first. The visual word is formed by clustering the image features of the images in the total learning data and setting the center of each cluster as the visual word. The image features can be local feature quantities such as SIFT or SURF commonly used in traditional feature bags, or they can be image features extracted using convolutional neural networks (CNNs). SIFT stands for Scale Invariant Feature Transform, and SURF stands for Speeded Up Robust Features.

[0109] Then, a histogram 831 is formed based on the images included in the learning dataset 801 and based on the visual words. The "intervals" of histogram 831 indicate the indices of the visual words. For convenience, Figure 8 The example shows the case where the number of visual word types is four. However, the number of visual word types is usually much larger. To form the histogram 831, image features are extracted from the images included in the training dataset 801, and visual words with the most similar image features are voted on. (Refer to...) Figure 8Arrow 822 indicates how image 821 voted for index 1, because the image features of image 821 are similar to the visual word of index 1. In a normal bag of features, histogram features are formed from a single image. However, the purpose of this embodiment is to form information indicating trends in the learning dataset. Therefore, image features are extracted from all images in the learning dataset 801 and voted for in the histogram.

[0110] Figure 8 The diagram illustrates how a vote 822 is performed from an image 821, since image 821 is an image with local block sizes. When the size of the learning image is large, local image features can also be extracted from an image, and the histogram can be voted on based on each image feature.

[0111] The above processing generates a histogram 831 of visual words. This histogram indicates the trend of image features in the learning dataset 801. That is, in this embodiment, by using each visual word as an attribute item, the data conditions of the learning dataset are expressed based on image features.

[0112] Then, as in the first embodiment, in order to form an attribute set in which binary information is set as "true" or "false" in each item, the histogram 831 is thresholded. Figure 8 The attribute set 841 formed by thresholding histogram 831 using threshold 832 is shown. In attribute set 841, only index 1 with a value greater than threshold 832 is "true". Note that it is also possible to form an attribute set in which the visual word index that has voted at least once is set to "true" without performing the thresholding process described above. However, when thresholding is performed by treating a small number of votes as noise, an attribute set for indicating the trend of the learning dataset can be correctly formed.

[0113] By performing the above processing on each learning dataset, an attribute set for the learning dataset is formed. As in the first embodiment, the information of these attribute sets is stored in the model storage unit 202 along with the model that has been learned using the learning dataset.

[0114] By using the same method, an attribute set based on image feature settings is also formed for the target data to be identified. Figure 9 A set of images 810 for identifying target data is shown. Based on this set of images, a histogram 833 is formed by using the same visual words used when forming histograms from the learning dataset, and information about the attribute set 851 is obtained.

[0115] The above processing yields attribute sets 841 to 84n for the learning dataset and attribute set 851 for the target data. The model selection method utilizing this information can be performed in the same manner as in the first embodiment, therefore its explanation will be omitted.

[0116] Please note that the visual words are automatically generated through clustering in the method described above, which makes it difficult to determine the image features indicated by each visual word. Since the visual words are attribute items in this embodiment, it is also difficult to determine the trends in the target data or learning dataset. Therefore, to clearly define the attributes indicated by the visual words, visual words for representing specific images can also be manually formed. For example, images with many cracks can be manually collected from the total learning data, and the distribution center of the feature values ​​of these images can be considered as a visual word. Thus, this visual word clearly indicates the attribute "many cracks".

[0117] As described above, this embodiment is based on an image-formed attribute set. This allows for the selection of the model to be used for recognition based on the image's appearance and environment.

[0118] Please note that a combination of human-understandable attribute items used in the first embodiment and image feature-based attributes used in the third embodiment can also be used. For example, in the first embodiment, attributes that can be explicitly input, such as structure type or camera parameters, can be set as attribute items with explicit names, and image-based attributes, such as wall state, can be formed based on image features. These attribute settings can also be combined to form an attribute set for model selection.

[0119] (Fourth Embodiment)

[0120] In the second embodiment, an example of selecting multiple models and presenting them to the user, with the user examining the selected models and making further model selections, has been explained. In the fourth embodiment, an example of presenting the model selection results to the user in this way will be explained in more detail below.

[0121] Please note that the hardware and functional configuration of the information processing apparatus 100 according to the fourth embodiment are the same as those of the first embodiment, and therefore their explanation will be omitted. Furthermore, as with the first embodiment, an embodiment using a crack identification model for inspecting infrastructure will also be explained in the fourth embodiment.

[0122] Figure 5This is a diagram used to explain the GUI application for model selection. The application window 900 includes areas 901 and 902; area 901 displays information about the target data for identification, and area 902 displays information about the model selected by the model selection unit 204. In this embodiment, the process from setting the attributes of the target data to selecting the model will be explained from the perspective of the user interface. The CPU 101 acts as a display control unit and displays the application window by controlling the display of the display unit 105.

[0123] Application window 900 has a region 911 for inputting attributes of the recognition target data. Region 911 is used for... Figure 9 The interfaces shown have the same purpose and function. However, in Figure 9 In the middle section, area 911 is the interface used to set "true" or "false" for each attribute item via drop-down buttons. After setting the attribute items, the user presses the best model search button 912.

[0124] In response to this operation, the model selection unit 204 selects a model suitable for identifying the target data. Figure 9 An example is shown where three types of models, M3, M8, and M6, are selected, and information for each model is displayed in its respective tab. Note that the order of the model tabs indicates the order of model diversity. Figure 9 Buttons 922 and 923 are shown for displaying these models by switching between them. Users switch between these models using buttons 922 and 923, and determine the model to be used in the recognition process while examining information about each model.

[0125] Figure 9 A confirmation button 924 is shown for selecting the currently displayed model as the model to be used in the recognition process. Note that the display of multiple models is not limited to this; multiple models can be displayed simultaneously and their contents compared. Furthermore, the number of models to be selected is not limited to one; multiple models can be selected. When multiple models are selected, recognition processing is performed using each model, the recognition results are compared, and the more suitable recognition result is determined.

[0126] The following will explain area 902, which displays model information. Area 902 displays information 925 about the model's attribute set. Users can compare and verify the information 925 about the model's attribute set and the information 911 about the attribute set of the target data, and select the model that has been learned under more similar conditions. Note that to make this comparison easier, attribute items with the same settings in the attribute set 911 of the target data and the attribute set 925 of the model can also be highlighted. Conversely, items with different settings can also be highlighted. At least one of the information 925 about the selected model's attribute set and the information 925 about the attribute set of the target data can also be displayed.

[0127] The area 902, which displays model information, can further show examples of learning images that have learned the model, as indicated in Figure 926. By displaying the learning images as information about the model in this way, users can more intuitively understand under what conditions the model was learned.

[0128] Please note that when attribute information is associated with each piece of learning data as in the first embodiment, only images for which predetermined attributes are set to "true" may be displayed. This allows users to easily understand, for example, the type of image learned from a given model that has the attribute "many cracks".

[0129] Additionally, area 901, which displays information about the identified target data, can also display an image of the identified target data. (See reference...) Figure 9 The area 901, which displays information about the target data, shows an image of the target data indicated by image 913. This image of the target data is, for example, an image of a portion of the structural wall of the target. By displaying the image of the target data and the image of the learned model in such a comparative manner, it is easier to select a model whose conditions are similar to those of the image of the target data.

[0130] Furthermore, instead of displaying all learned images, typical images for each attribute can be pre-selected and displayed as images 926 that have already been learned from the model. If the user can understand the trends in the images learned by the model, similar sample images, legends, etc., can be used instead of images that were not actually learned. This allows images that can be easily compared to images used to identify the target data to be displayed.

[0131] The image 926 that has been learned by the model does not need to be an actual learned image; it can also be a virtually generated image. Recently, a technique called GAN (Generative Adversarial Network) has been developing a method for generating images based on learning. Using this technique, trends in images within a learning dataset can be learned, images similar to those in the learning dataset can be generated using the learned model, and the generated images can be used as learning images for display. It is also possible to display at least one of the image 926, which represents the learning data of a selected model, and the image 913, representing the target recognition data.

[0132] As described above, when an interface is provided for selecting one or more models from a plurality of models selected by the model selection unit 204, the user can easily select a model. Note that the example of selecting a model to be used in the recognition process from multiple models has already been explained above. However, even when the model selection unit 204 selects only one model, a user interface according to this embodiment can still be provided to check the selected model. In this case, it is not necessary to display multiple models by switching models. Figure 10 Buttons 922 and 923, and button 924 for selecting the model.

[0133] The CPU 101 of the information processing device 100 performs the above-mentioned processing by loading a program stored in ROM 102 into RAM 103 and executing the program. Furthermore, the CPU 101 displays a GUI on the display unit 105 and accepts user input via the operation unit 106 as needed.

[0134] Model selection and recognition processing can also be carried out in the form of SaaS (Software as a Service). Figure 10 Information processing devices 110, 120, 121, and 122 in SaaS form are shown. Like information processing device 100, each of these information processing devices includes a CPU 101, ROM 102, RAM 103, HDD 104, display unit 105, operation unit 106, and communication unit 107. The construction of each information processing device is the same as that of information processing device 100, therefore detailed explanations will be omitted.

[0135] exist Figure 9 In the example shown, information processing device 110 is the server, and information processing devices 120, 121, and 122 are clients. Client information processing devices 120, 121, and 122 receive information from server information processing device 110 and perform processing related to the user interface. More specifically, each client displays as explained in this embodiment. Figure 10The GUI shown accepts user input. The server information processing unit 110 performs the model selection and recognition processing functions explained in the first embodiment. Note that the number of client information processing units is not limited to... Figure 11 The number of examples shown. By providing the service as a SaaS in this way, the service can be offered to many users.

[0136] (Fifth Embodiment)

[0137] In the fifth embodiment, an example of adding model evaluation as a criterion for determining model selection is explained.

[0138] Figure 2 This is a block diagram example illustrating the functional structure of the information processing apparatus 100 according to the fifth embodiment. The CPU 101 implements these functional units by extending the program stored in ROM 102 into RAM 103 and executing processes conforming to the respective flowcharts. The CPU 101 stores the execution results of each process in RAM 103 or HDD 104. When, for example, hardware is constructed instead of software processing using the CPU 101, only the circuitry and arithmetic units corresponding to the processing of the functional units to be explained herein need to be constructed. Note that the following explanation includes embodiments in the form of SaaS. However, in the form of SaaS, the server information processing apparatus and the client information processing apparatus perform the following processes as explained in the fourth embodiment.

[0139] Apart from Figure 12 In addition to the functional block diagram shown, the information processing apparatus 100 according to this embodiment also includes a model evaluation unit 206. The model evaluation unit 206 performs model evaluation and collects information. The processing to be performed by the model evaluation unit 206 will be explained below.

[0140] One embodiment of model evaluation is using a method of user evaluation of the model. As explained in the fourth embodiment, the service can be provided to many users in the form of SaaS. Each user receives the recognition results using a selected model and is satisfied with the results in some cases and dissatisfied in others. In this embodiment, user satisfaction is collected for each model and used as an evaluation of the model.

[0141] Figure 12 This is a view used to interpret the interface used to collect user satisfaction data for each model. The client information processing device presents the following to the user: Figure 12 The interface shown is as follows. GUI window 1200 displays the recognition result 1201 of the target image identified by the given model. Figure 12 In the example shown, recognition result 1201 is the crack recognition result for a given structural wall image. The user sees this result and enters their satisfaction level with the recognition result.Figure 13 An input area 1202 for inputting user satisfaction is shown. In this example, selections can be made using radio buttons for evaluation on three levels: "Good," "Neutral," and "Bad." The user inputs one of these evaluations and presses the send button 1203. Thus, the model forming the recognition result and the user evaluation are correlated and recorded in the server information processing device.

[0142] The evaluation value (evaluation accuracy) of each model is determined by accumulating user evaluations as described above. The evaluation value of a model can be obtained simply by rating and accumulating the user evaluations. For example, user evaluations of each model are accumulated by setting "good" to +1, "average" to 0, and "poor" to -1. Therefore, models with high user satisfaction and high performance have higher evaluation values. As described above, the model evaluation unit 206 performs the collection and accumulation of user evaluations and the calculation of model evaluation values.

[0143] In the model selection process, the model selection unit 204 obtains the evaluation values ​​of each model from the model evaluation unit 206 and uses these evaluation values ​​as the criteria for model selection. That is, the model selection unit 204 selects models based on user evaluations. For example, in step S602 of the first embodiment, the model selection unit 204 selects models with low diversity values ​​and high evaluation values. By taking model evaluation values ​​into account when selecting models, validated models with high user satisfaction can be chosen.

[0144] As another method for measuring user satisfaction, the number of times a model is selected can also be recorded. As explained in the second and fourth embodiments, the model selection unit 204 can also select multiple models and present them to the user. In this case, the user selects the model to be used in the recognition process from among the multiple models. The selection count is recorded for each model, and these records are summed, so that the model evaluation value increases as the selection count increases.

[0145] Another method for measuring user satisfaction is as follows. That is, in order to enable users to reuse the selected model, they can also register the model through bookmarks and calculate the model evaluation value based on the registration count.

[0146] The example of using user satisfaction-based model evaluation in model selection has been explained above.

[0147] As another embodiment of the model selection method using model evaluation, the method of evaluating a model using prepared performance evaluation data will be explained below. In this embodiment, similar to the total learning data in the first embodiment, a large amount of performance evaluation data is prepared as total performance evaluation data. This large amount of performance evaluation data contains pairs of images and ground truth information indicating the location of cracks in the images. Furthermore, information indicating the attributes of each performance evaluation data point is associated with and recorded. The images in the aforementioned performance evaluation data are processed using a given model, and the processing result is compared with the ground truth information. Therefore, the accuracy or precision of the model can be obtained. In embodiments where the performance evaluation results are used as evaluation values, the accuracy or precision obtained as described above is used as the evaluation value.

[0148] Furthermore, a performance evaluation dataset is formed that matches the attribute set of the target data. This dataset undergoes model-based identification processing to calculate accuracy or precision and determine the model's evaluation value. Then, performance evaluation data is collected from the total performance evaluation data to create a dataset whose attribute set is similar to that of the target data. This processing can be performed on all models or only on a limited number of models whose attribute sets are similar to those of the target data. Evaluation values ​​are determined by calculating accuracy or precision relative to the performance evaluation dataset.

[0149] The evaluation value of the model calculated as described above is used in model selection. Model selection using this evaluation value can be performed in the same manner as when using the aforementioned user-evaluated evaluation value.

[0150] Please note that the above explanation has already described an example of constructing an evaluation dataset and evaluating the model based on the attribute set of the target data. However, evaluation values ​​can also be calculated by evaluating the model using any pre-set performance evaluation data. In this case, various performance evaluation datasets with different attribute sets are formed, and evaluation values ​​are calculated by evaluating each model using each performance evaluation dataset in these datasets. The performance evaluation datasets and models are then recorded as being associated with each other. In the model selection process, the attributes of the target data are set, and when the attribute set of the target data is obtained, the evaluation values ​​evaluated using a performance evaluation dataset with an attribute set similar to that of the target data are called as the evaluation values ​​for each model. The called evaluation values ​​are then used as the evaluation values ​​for each model in the model selection process.

[0151] As mentioned above, when constructing the performance evaluation dataset and pre-evaluating the model, model evaluation is not required whenever the attributes of the target data are set, so the time required for evaluation and model selection can be omitted.

[0152] Please note that if a suitable model for identifying the target data cannot be selected after the processing of selecting a model based on the attributes of the target data, a new model can be learned. For example, if a model that includes the attribute set of the target data cannot be selected from the existing learning models, the learning unit 201 performs the processing of learning a new model.

[0153] Learning unit 201 forms a learning dataset for a new model to be learned based on the attribute set of the target data. More specifically, data for the learning dataset is collected from the total learning data to obtain an attribute set similar to the attribute set of the target data. In the process of forming a learning dataset similar to the attribute set of the target data, at least an attribute set of the learning dataset including the attribute set of the target data can be constructed. A new model is learned using the learning dataset formed as described above. Therefore, a model suitable for the target data can be formed.

[0154] Please note that when learning a model based on the target data, additional learning can be performed on an existing learning model to reduce learning time. This additional learning is performed by using the model selected in the above embodiments as the base model. More specifically, a model suitable for recognizing the target data is first selected from existing learning models through the process explained in this embodiment or other embodiments. Using this model as the base model can shorten the learning time. Furthermore, as the learning dataset used in the additional learning, a learning dataset with an attribute set similar to the attribute set of the target data is formed as described above. Additionally, the learning dataset can be constructed by comparing the attribute set of the model used as the base model with the attribute set of the target data and collecting only learning data with attributes missing in the recognition of the target data.

[0155] (Sixth Embodiment)

[0156] In the above embodiments, the identification of cracks in infrastructure inspection has been used as an example of the identification target. However, the invention is not limited thereto and can be used in other applications. In the sixth embodiment, the selection of an identification model to be used in a fixed camera, such as a surveillance camera, will be explained. Note that the hardware and functional configuration of the information processing apparatus 100 according to the sixth embodiment are the same as those of the first to fifth embodiments, and therefore their explanation will be omitted.

[0157] In this embodiment, a model suitable for an environment with a fixed camera, such as a surveillance camera, is selected from multiple learning models. In this case, the attributes for identifying the target, as explained in the first to fifth embodiments, only need to be changed to attributes related to the identification performed by the fixed camera. Apart from the attribute items, the same processing as explained in the first to fifth embodiments can be performed. Therefore, the attributes used in embodiments where the fixed camera is the target will be explained below.

[0158] Reference Figure 13 Explain the attribute items in this embodiment. Figure 13 The user interface is shown for setting attributes of the target data to select a fixed camera model. Figure 5 This is the first embodiment. Figure 13 Variations. Examples of attributes in embodiments with fixed cameras include items related to the camera mounting environment, target identification, and imaging. Figure 13 Examples are shown that include attribute items "Shop," "Station," and "Road" as camera mounting environments. The primary target to be identified within the camera mounting environment can also be set as an attribute item, and... Figure 13 An example is shown that includes the attribute items "Person" and "Vehicle". Please note that... Figure 13 The attribute items shown are merely examples, and therefore the invention is not limited to these attribute items; other attribute items may also be included as part of the camera mounting environment. For example, items such as "internal," "external," and "nighttime imaging" may also be included. Please note that... Figure 13 This shows the user setting "True" in "Shop" and "False" in "Station," and the user continues to set attributes for other attribute items, such as the fixed camera used as the model selection target. Additionally, as... ​ As shown, you can also set "invalid" in the attribute item as in the first embodiment.

[0159] As described above, when setting the attributes of the target data for identification, these attributes are set for the fixed camera that serves as the target for model selection. When all attribute items are fully set through this process, an attribute set of the fixed camera image, which serves as the target data for identification, is formed. The process of selecting a learning model based on the attribute set of the target data for identification can be performed in the same manner as in other embodiments. Note that for this process, each learning model and attribute set information indicating the combination of attributes used in the learned data of the model are stored in a way that is related to each other.

[0160] As described above, the information processing apparatus according to the above embodiments can be used to perform model selection for any identification target by changing attribute items.

[0161] (Other embodiments)

[0162] This invention can be implemented by supplying a program that performs one or more functions of the above embodiments to a system or device via a network or storage medium, and having one or more processors in the computer of the system or device read and execute the program. This invention can also be implemented by circuitry (e.g., an ASIC) for performing one or more functions.

[0163] This invention is not limited to the embodiments described above, and various changes and modifications can be made within the spirit and scope of this invention. Therefore, the appended claims are made to inform the public of the scope of this invention.

[0164] This application claims priority to Japanese Patent Application No. 2019-206352, filed on November 14, 2019, which is incorporated herein by reference.

Claims

1. An information processing apparatus, the information processing apparatus comprising: The first acquisition unit is configured to acquire information about the learning data and use the learning data in the learning of each of the multiple pre-learning models used to identify the input data. The second acquisition unit is configured to acquire information indicating attributes of identification target data, said identification target data including images of infrastructure; as well as A model selection unit is configured to select one or more models from a plurality of models to be used for recognizing the target data, based on the degree of matching between the attributes of the target data and the attributes of the learning data used in the learning of each of the plurality of models, and based on the diversity of the attributes of the learning data used in the learning of each of the plurality of models, wherein the one or more models have been learned using learning data having attributes including those of the target data, and the attribute items of the learning data match the attribute items of the target data that are set to "true". In cases where the model selection unit selects more than one model, the model selection unit chooses from the selected models the model that was learned using training data with low diversity. The attributes of the learning data and the attributes of the target data are information that sets "true" or "false" for each attribute item. The identification of the target data includes the identification of defects in the infrastructure, and Wherein, when the model selection unit selects several models that have learned the attribute items set to "true", the model selection unit is configured to select from the several models the model with the smallest number of attribute items set to "true" in the learning data as the model to be used for the recognition of the recognition target data.

2. The information processing apparatus according to claim 1, wherein, Diversity is represented by the number of attributes in the learning data.

3. The information processing apparatus according to claim 1, further comprising: The display control unit is configured to control the display unit to display at least one of the attributes of the identified target data and the attributes of the model selected by the model selection unit.

4. The information processing apparatus according to claim 3, wherein, The display control unit controls the display unit to further display at least one of the image of the identified target data and the image of the learning data of the model selected by the model selection unit.

5. The information processing apparatus according to claim 1, wherein, The model selection unit further selects models based on their recognition accuracy.

6. The information processing apparatus according to claim 1, wherein, The model selection unit further selects models based on user evaluations of the models.

7. The information processing apparatus according to claim 1, wherein, Attributes are image features based on the image.

8. The information processing apparatus according to claim 1, wherein, The learning data includes multiple data sets used for learning, and The attributes of the learning data are determined based on the sum of the attributes of the multiple data used for learning.

9. A computer-readable storage medium storing a program that enables a computer to function as an information processing apparatus as defined in any one of claims 1 to 8.

10. An information processing method, the information processing method comprising: Obtain information about the learning data and use the learning data in the learning of each of the multiple pre-learning models used to identify the input data; Acquire information indicating attributes of target data to be identified, the target data including images of infrastructure; Based on the matching degree between the attributes of the target data and the attributes of the learning data used in the learning of each of the multiple models, and based on the diversity of the attributes of the learning data used in the learning of each of the multiple models, one or more models are selected from the multiple models to be used for the recognition of the target data, wherein the one or more models have been learned using learning data having attributes including those of the target data, and the attribute items of the learning data match the attribute items of the target data that are set to "true". In cases where more than one model is selected, the model that was learned using training data with low diversity is chosen from among the selected models. The attributes of the learning data and the attributes of the target data are information that sets "true" or "false" for each attribute item. The identification of the target data includes the identification of defects in the infrastructure; and In the case of selecting several models that have learned the attribute items set to "true", the model with the smallest number of attribute items set to "true" in the learning data is selected from the several models as the model to be used for the recognition of the target data.

Images