Information processing device, information processing method, method for producing detection model, and program

The information processing device automates threshold determination for pseudo labels, reducing costs and improving the efficiency of generating highly accurate detection models by directly associating reliable inference results with correct labels.

JP7582442B2Active Publication Date: 2024-11-13NEC CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023503690
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-03-05
Filing Date
2022-02-15
Publication Date
2024-11-13
Estimated Expiration
2042-02-15

AI Technical Summary

Technical Problem

Existing methods for generating highly accurate detection models using pseudo labels require time-consuming and costly adjustments to set appropriate threshold values, leading to inefficiencies in the generation process.

Method used

An information processing device that automatically determines threshold values by comparing inference results with correct labels, setting reliable inference results as pseudo labels, and generating datasets for relearning detection models, thereby reducing the need for manual adjustments.

Benefits of technology

This approach allows for the generation of highly accurate detection models while minimizing the time and computational costs associated with threshold adjustments, enhancing the efficiency of the model generation process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007582442000004
    Figure 0007582442000004
  • Figure 0007582442000005
    Figure 0007582442000005
  • Figure 0007582442000006
    Figure 0007582442000006
Patent Text Reader

Abstract

In order to generate a high-accuracy detection model while suppressing generation costs, this information processing device (10) comprises a training unit (101), a threshold value determination unit (102), an inference unit (103), and a dataset generation unit (104). The training unit trains a detection model using a first dataset. The threshold value determination unit compares an inference result obtained by inputting an image included in a dataset for evaluation to the detection model and a correct-answer label attached to the image, thereby determining a threshold value. The inference unit inputs an image included in a second dataset to the detection model and acquires an inference result for the image. The dataset generation unit: associates an inference result that is produced by the inference unit and that has a reliability greater than or equal to the threshold value with the corresponding image, the inference result being associated as a pseudo-label; and generates a dataset after addition of the pseudo-label.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to techniques for associating pseudo labels with one or more images included in a dataset used to retrain a detection model. [Background technology]

[0002] A detection model that detects objects contained in an image can become highly accurate by learning using a large amount of correct answer data. However, the process of collecting a large amount of data and associating correct answer labels with the data is costly. For this reason, a technique is known that associates pseudo labels with non-correct answer data in order to generate a highly accurate detection model from a small amount of correct answer data.

[0003] A pseudo label refers to a reliable inference result obtained by inferring an image from a dataset without a correct answer using a detection model trained only on a dataset with a correct answer. For example, Non-Patent Document 1 discloses a method of adopting an inference result whose reliability is equal to or exceeds a threshold as a pseudo label. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Xiangyun Zhao et. al.,"Object Detection with a Unified Label Space from Multiple Datasets",August 15, 2020,arXiv:2008.06614v1 Summary of the Invention [Problem to be solved by the invention]

[0005] The method described in Non-Patent Document 1 requires adjustment to set an appropriate threshold, so there is room for reducing the time and computational costs involved in this adjustment. In other words, there is room for further reducing the cost of generating a highly accurate detection model using pseudo labels.

[0006] One aspect of the present invention has been made in consideration of the above problems. That is, an object of one aspect of the present invention is to provide a technique capable of generating a highly accurate detection model while suppressing generation costs. [Means for solving the problem]

[0007] An information processing device according to one embodiment of the present invention includes: a learning means for learning a detection model using a first dataset; a threshold determination means for determining a first threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more correct labels assigned to each of the one or more images; an inference means for inputting each of one or more images included in a second dataset into the detection model, thereby obtaining one or more inference results for each of the one or more images; and a dataset generation means for generating a dataset after pseudo-labeling by setting an inference result having a reliability equal to or higher than the first threshold among the one or more inference results by the inference means as a pseudo label and associating the pseudo label with a corresponding image.

[0008] An information processing device according to one aspect of the present invention includes a first learning means for learning a first detection model using a first dataset, a second learning means for learning a second detection model using a second dataset, a first threshold determination means for determining a first threshold by referring to a result of comparison between one or more inference results obtained by inputting each of one or more images included in a first evaluation dataset into the first detection model and one or more correct labels attached to each of the one or more images, and a second threshold determination means for determining a second threshold by referring to a result of comparison between one or more inference results obtained by inputting each of one or more images included in a second evaluation dataset into the second detection model and one or more correct labels attached to each of the one or more images. the first inference means for acquiring one or more inference results for each of one or more images; the second inference means for acquiring one or more inference results for each of the one or more images included in the first dataset by inputting each of the one or more images to the second detection model; the first dataset generation means for generating a second dataset after pseudo-labeling by setting, among the one or more inference results by the first inference means, an inference result having a reliability equal to or greater than the first threshold as a pseudo label and associating the pseudo label with a corresponding image; and the second dataset generation means for generating the first dataset after pseudo-labeling by setting, among the one or more inference results by the second inference means, an inference result having a reliability equal to or greater than the second threshold as a pseudo label and associating the pseudo label with a corresponding image.

[0009] an information processing device according to one aspect of the present invention, the information processing device including: an acquisition means for acquiring a target image; and a detection means for detecting an object included in the target image using a detection model for the target image, the detection model for the target image being trained through a learning process for training a detection model using a first dataset; a threshold determination process for determining a first threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more ground truth labels attached to each of the one or more images; an inference process for inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; a dataset generation process for generating a dataset after pseudo-labeling by setting an inference result having a reliability equal to or higher than the first threshold among the one or more inference results obtained by the inference process as a pseudo label and associating the pseudo label with a corresponding image; and a pseudo-label reference learning process for training the detection model for the target image by referring to the dataset after pseudo-labeling.

[0010] An information processing method according to one embodiment of the present invention includes a learning step of learning a detection model using a first dataset; a threshold determination step of determining a first threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more correct labels assigned to each of the one or more images; an inference step of inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; and a dataset generation step of setting an inference result having a reliability equal to or higher than the first threshold as a pseudo label from among the one or more inference results obtained by the inference step, and associating the pseudo label with a corresponding image, thereby generating a dataset after the pseudo label has been assigned.

[0011] an information processing method according to one aspect of the present invention, comprising: acquiring a target image; and detecting an object included in the target image using a detection model for the target image, wherein the detection model for the target image is trained through a learning process for training a detection model using a first dataset; a threshold determination process for determining a first threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more ground truth labels assigned to each of the one or more images; an inference process for inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; a dataset generation process for generating a dataset after pseudo-labeling by setting an inference result having a reliability equal to or higher than the first threshold among the one or more inference results obtained by the inference process as a pseudo label and associating the pseudo label with a corresponding image; and a pseudo-label reference learning process for training the detection model for the target image by referring to the dataset after pseudo-labeling.

[0012] A method for manufacturing a detection model according to one embodiment of the present invention includes: a learning step of learning a detection model using a first dataset; a threshold determination step of determining a first threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more correct labels assigned to each of the one or more images; an inference step of inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; a dataset generation step of generating a pseudo-labeled dataset by setting an inference result having a reliability equal to or higher than the first threshold among the one or more inference results obtained by the inference step as a pseudo label and associating the pseudo label with a corresponding image; and a pseudo-label reference learning step of training a target image detection model for detecting an object included in the target image using the pseudo-labeled dataset.

[0013] A program according to one aspect of the present invention is a program for causing a computer to function as an information processing device, and causes the computer to function as: a learning means for learning a detection model using a first dataset; a threshold determination means for determining a threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more correct labels assigned to each of the one or more images; an inference means for inputting each of one or more images included in a second dataset into the detection model, thereby obtaining one or more inference results for each of the one or more images; and a dataset generation means for generating a dataset after the pseudo labels have been assigned, by setting an inference result having a reliability equal to or higher than the threshold among the one or more inference results obtained by the inference means as a pseudo label and associating the pseudo label with a corresponding image.

[0014] a threshold determination process for determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model and one or more correct labels assigned to each of the one or more images; an inference process for acquiring one or more inference results for each of the one or more images included in a second dataset into the detection model; a dataset generation process for generating a dataset after pseudo-labeling by setting an inference result having a reliability equal to or higher than the threshold among the one or more inference results obtained by the inference process as a pseudo label and associating the pseudo label with a corresponding image; and a pseudo-label reference learning process for learning the detection model for the target image by referring to the dataset after pseudo-labeling. Effect of the Invention

[0015] According to one aspect of the present invention, it is possible to generate a highly accurate detection model while suppressing generation costs. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment of the present invention. [Diagram 2] 2 is a flowchart showing the flow of an information processing method executed by the information processing device shown in FIG. [Diagram 3] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment of the present invention. [Figure 4] 4 is a flowchart showing the flow of an information processing method executed by the information processing device shown in FIG. [Diagram 5] FIG. 11 is a block diagram showing a configuration of an information processing device according to a second exemplary embodiment of the present invention. [Figure 6] FIG. 11 is a diagram showing a specific example of data included in a first data set and a second data set according to an exemplary embodiment 2 of the present invention. [Figure 7] 6 is a graph showing the relationship between the precision rate and the recall rate calculated by the information processing device shown in FIG. 5 . [Figure 8] FIG. 11 is a diagram showing a specific example of data included in a pseudo-labeled dataset according to an exemplary embodiment 2 of the present invention. [Figure 9] 6 is a flowchart showing the flow of an information processing method executed by the information processing device shown in FIG. 5. [Figure 10] FIG. 11 is a block diagram showing a configuration of an information processing device according to a second exemplary embodiment of the present invention. [Figure 11] FIG. 11 is a block diagram showing a configuration of an information processing device according to an exemplary embodiment 3 of the present invention. [Figure 12] FIG. 11 is a diagram showing a specific example of data included in a first data set and a second data set according to an exemplary embodiment 3 of the present invention. [Figure 13] FIG. 11 is a diagram showing a specific example of data contained in a second dataset according to an exemplary embodiment 3 of the present invention, and a pseudo-labeled dataset generated from the second dataset. [Figure 14] FIG. 11 is a block diagram showing a configuration of an information processing device according to an exemplary embodiment 4 of the present invention. [Figure 15] FIG. 11 is a diagram showing a specific example of data included in a first data set and a second data set according to an exemplary embodiment 4 of the present invention. [Figure 16] FIG. 11 is a diagram showing a specific example of data included in a pseudo-labeled dataset according to an exemplary embodiment 4 of the present invention. [Figure 17] FIG. 11 is a block diagram showing a configuration of an information processing device according to an exemplary embodiment 5 of the present invention. [Figure 18] 18 is a flowchart showing the flow of an information processing method executed by the information processing device shown in FIG. 17. [Figure 19] FIG. 13 is a block diagram showing a configuration of an information processing device according to an exemplary embodiment 6 of the present invention. [Figure 20] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device in each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Example Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later.

[0018] <Overview of information processing device 10> The information processing device 10 according to this exemplary embodiment has a function as a dataset generating device that generates a pseudo-labeled dataset by adding pseudo labels to a target dataset.

[0019] More specifically, the information processing device 10 first learns a detection model using a first dataset. Furthermore, the information processing device 10 determines a first threshold value by referring to a comparison result between one or more inference results obtained by inputting one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the images. Furthermore, the information processing device 10 inputs one or more images included in the second dataset into the detection model, thereby obtaining one or more inference results for each of the images. Furthermore, the information processing device 10 sets an inference result having a reliability equal to or higher than a first threshold value, among one or more inference results from one or more images included in the second dataset, as a pseudo label, and generates a pseudo-labeled dataset by associating the pseudo label with a corresponding image.

[0020] <Configuration of information processing device 10> The configuration of an information processing device 10 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 10.

[0021] As shown in Fig. 1, the information processing device 10 includes a learning unit 101, a threshold determination unit 102, an inference unit 103, and a data set generation unit 104. The learning unit 101 is configured to realize a learning means in this exemplary embodiment. The threshold determination unit 102 is configured to realize a threshold determination means in this exemplary embodiment. The inference unit 103 is configured to realize an inference means in this exemplary embodiment. The data set generation unit 104 is configured to realize a data set generation means in this exemplary embodiment.

[0022] The learning unit 101 uses a first data set to learn a detection model. Specifically, the learning unit 101 uses a first data set including one or more images to learn a detection model for detecting an object included in the image. Detection refers to inputting an image to the detection model, - The presence or absence of objects in the image - The position of the object contained in the image - The size of the object contained in the image The category of objects contained in the image The learning unit 101 receives an image as an input and causes a detection model to learn, which outputs the above-mentioned inference results.

[0023] The threshold determination unit 102 determines the first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the images. Here, the correct label is a label that indicates, for one or more objects included in each of one or more images included in the evaluation dataset, - The position of the object contained in the image - The size of the object contained in the image The category of objects contained in the image A label that contains ground truth data regarding at least one of the above.

[0024] The inference unit 103 inputs each of one or more images included in the second data set into the above-mentioned detection model to obtain one or more inference results for each of the images. The second data set includes one or more images different from the first data set.

[0025] The dataset generation unit 104 sets an inference result having a reliability equal to or higher than a first threshold value, among one or more inference results by the inference unit 103, as a pseudo label, and associates the pseudo label with a corresponding image, thereby generating a pseudo-labeled dataset. Here, the pseudo label is defined as a label for each of one or more images included in the second dataset. Each position of one or more objects inferred to be an object by the inference unit 103 -Size of each object above - Each of the above object categories A label including data relating to at least one of the above.

[0026] In addition, assuming that a correct label exists for a certain object, the pseudo label given to the object may or may not match the correct label. For example, it may happen that one or more of the position, size, and category of the object included in the correct answer data for the object match the position, size, and category of the object in the pseudo label, but the other items do not match.

[0027] The accuracy of the pseudo labels can generally be adjusted by adjusting the first threshold mentioned above, but adjusting the first threshold may generally require time and computational costs.

[0028] As described above, the information processing device 10 according to the present exemplary embodiment employs a configuration for automatically determining a first threshold for determining whether or not to treat an inference result for each image included in the second data set as a pseudo label. Therefore, the information processing device 10 according to the present exemplary embodiment has an effect of reducing the cost associated with adjusting the first threshold. Therefore, the information processing device 10 according to the present exemplary embodiment can generate a highly accurate detection model while suppressing generation costs.

[0029] <Information processing method flow> The flow of the information processing method S10 executed by the information processing device 10 configured as above will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S10. The information processing device 10 executes the information processing method S10 to generate a second data set including images associated with pseudo labels.

[0030] As shown in FIG. 2, the information processing method S10 includes steps S101 to S104.

[0031] (Step S101) In step S101, the learning unit 101 learns a detection model. Specifically, the learning unit 101 learns the detection model using a first data set. Step S101 is a learning step in this exemplary embodiment.

[0032] (Step S102) In step S102, the threshold determination unit 102 determines a first threshold. Specifically, the threshold determination unit 102 determines a first threshold for determining a pseudo label by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into a detection model and one or more correct labels attached to each of the images. Step S102 is a threshold determination step in this exemplary embodiment.

[0033] (Step S103) In step S103, the inference unit 103 performs inference. Specifically, the inference unit 103 inputs each of one or more images included in the second data set into a detection model, thereby obtaining one or more inference results for each of the images. Step S103 is an inference step in this exemplary embodiment.

[0034] (Step S104) In step S104, the dataset generator 104 generates a dataset after pseudo-labeling. Specifically, the dataset generator 104 sets an inference result having a reliability equal to or higher than a first threshold, among one or more inference results in step S103, as a pseudo label, and associates the pseudo label with a corresponding image in the second dataset, thereby generating a dataset after pseudo-labeling. Step S104 is a dataset generating step in this exemplary embodiment.

[0035] The execution timing of step S103 is not limited to after step S102, but may be any time after step S101 and before step S104, for example, before step S102.

[0036] As described above, the information processing method S10 according to this exemplary embodiment can provide the same effect as the information processing device 10. That is, the information processing method S10 according to this exemplary embodiment employs a configuration for automatically determining a first threshold for determining whether or not to treat the inference result for each image included in the second data set as a pseudo label. Therefore, the information processing method S10 according to this exemplary embodiment can provide the effect of reducing the cost associated with adjusting the first threshold. Therefore, the information processing method S10 according to this exemplary embodiment can generate a highly accurate detection model while suppressing the generation cost.

[0037] <Overview of information processing device 20> The information processing device 20 acquires a target image, and detects an object included in the image using a detection model for the target image. Typically, the detection model for the target image is a detection model that is re-learned with respect to the detection model learned by the above-mentioned information processing device 10, specifically the learning unit 101, and is a detection model that has been re-learned by referring to a dataset after pseudo-labeling generated by the information processing device 10. Note that the detection model for the target image is not limited to this. The detection model for the target image may be a detection model learned using a dataset after pseudo-labeling, for example, a new detection model learned using a dataset after pseudo-labeling. Here, the new detection model is a detection model different from the detection model learned by the learning unit 101.

[0038] <Configuration of information processing device 20> The configuration of the information processing device 20 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 20.

[0039] 3, the information processing device 20 includes an acquisition unit 201 and a detection unit 202. The acquisition unit 201 is configured to realize an acquisition unit in this exemplary embodiment. The detection unit 202 is configured to realize a detection unit in this exemplary embodiment.

[0040] The acquisition unit 201 acquires a target image. Here, the target image is an image input to a detection model in order to detect an object included in the image. For example, the acquisition unit 201 may acquire the target image by reading out the target image stored in the information processing device 20, or may acquire the target image supplied from an imaging device. For example, the acquisition unit 201 may acquire the target image via an input device (not shown). For example, the acquisition unit 201 may acquire the target image from another device (not shown) communicably connected to the information processing device 20.

[0041] The detection unit 202 detects an object included in a target image using a detection model for a target image. The detection model for a target image is a detection model used to detect an object included in a target image, and the detection model for a target image according to this exemplary embodiment is a detection model that has been re-learned as described above. The detection unit 202 inputs a target image to the detection model for a target image, thereby acquiring an inference result output from the detection model for a target image. For example, the detection unit 202 holds a detection model for a target image, and inputs a target image to the detection model for a target image. Also, for example, the detection unit 202 accesses the detection model for a target image stored in a storage device (not shown), and inputs the target image.

[0042] As described above, the information processing device 20 according to the present exemplary embodiment employs a configuration in which a pseudo label is determined using an automatically determined first threshold, and an object is detected using a target image detection model trained using a dataset including an image associated with the pseudo label. Therefore, the information processing device 20 according to the present exemplary embodiment has an effect of being able to detect an object included in an image using a target image detection model that reduces the cost associated with adjusting the first threshold.

[0043] <Information processing method flow> The flow of the information processing method S20 executed by the information processing device 20 configured as above will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing the flow of the information processing method S20. The information processing device 20 executes the information processing method S20 in order to detect an object included in a target image.

[0044] As shown in FIG. 4, the information processing method S20 includes steps S201 and S202.

[0045] (Step S201) In step S201, the acquisition unit 201 acquires a target image.

[0046] (Step S202) In step S202, the detection unit 202 detects an object. Specifically, the detection unit 202 detects an object included in the target image using a detection model for the target image. More specifically, the detection unit 202 inputs the target image acquired by the acquisition unit 201 to the detection model for the target image, and acquires an inference result output by the detection model. As described above, the information processing method S20 according to this exemplary embodiment provides the same effect as the information processing device 20. That is, the information processing method S20 according to this exemplary embodiment employs a configuration in which a pseudo label is determined using an automatically determined first threshold, and an object is detected using a target image detection model trained using a dataset including an image associated with the pseudo label. Therefore, the information processing method S20 according to this exemplary embodiment provides the effect of being able to detect an object included in an image using a target image detection model in which the cost associated with adjusting the first threshold is reduced.

[0047] Exemplary embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0048] <Overview of the information processing device 10a> The information processing device 10a according to this exemplary embodiment is a modification of the exemplary embodiment 1. Specifically, the information processing device 10a acquires a first dataset, and performs learning of the detection model described in the exemplary embodiment 1, determining a threshold, making inference, and creating a dataset after pseudo-labeling. Furthermore, the information processing device 10a uses the generated dataset after pseudo-labeling to learn a detection model for a target image. Typically, the detection model for a target image is a detection model that is re-trained with respect to the above detection model, and is re-trained by referring to the dataset after pseudo-labeling. As described above, the detection model for a target image is not limited to a detection model that has been re-trained, and may be a detection model that has been trained using a dataset after pseudo-labeling.

[0049] <Configuration of information processing device 10a> The configuration of the information processing device 10a will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the information processing device 10a. As shown in Fig. 5, the information processing device 10a includes a control unit 100a and a storage unit 150a. The control unit 100a controls each unit of the information processing device 10a. The storage unit 150a stores various programs and data used by the information processing device 10a.

[0050] The storage unit 150a stores an evaluation dataset DSE, dataset 1 (DS1), dataset 2 (DS2), dataset 2' (DS2'), and an object detection model DM. The evaluation dataset DSE is an evaluation dataset in this exemplary embodiment. The dataset 1 (DS1) is a first dataset in this exemplary embodiment. The dataset 2 (DS2) is a second dataset in this exemplary embodiment. The dataset 2' (DS2') is a dataset after pseudo-labeling in this exemplary embodiment.

[0051] Here, the details of dataset 1 (DS1) and dataset 2 (DS2) will be described. Fig. 6 is a diagram showing a specific example of data included in dataset 1 (DS1) and dataset 2 (DS2). Specifically, Fig. 6 shows one of the images included in dataset 1 (DS1) and one of the images included in dataset 2 (DS2).

[0052] Each of these images includes five objects, specifically, three people and two bags. In the images included in Data Set 1 (DS1), each of the five objects is associated with a correct label. Typically, the correct label is a label including a category and a bounding box, as shown in FIG. 6. The category is category information indicating the category of an object included in the image associated with the correct label, and specifically, is correct data regarding the category of the object. In the example of FIG. 6, the category of "person" is associated with each of the three people, and the category of "bag" is associated with each of the two bags. The bounding box is area information indicating the area of ​​an object included in the image associated with the correct label, and specifically, is correct data regarding the position and size of the object included in the image. One bounding box is associated with one object, and a typical example of the bounding box is data indicating the smallest rectangle in which the object is contained, as shown in FIG. 6.

[0053] On the other hand, in the images in Dataset 2 (DS2), no ground truth labels are associated with the objects.

[0054] Based on the above, the "images", "datasets", and "correct labels" described in each exemplary embodiment can be expressed as follows: An image x input to the detection model is an element of a data space X, where X corresponds to a dataset that contains the image x. Note that an image x can contain any number of objects. The correct label can be expressed as a pair (y, b) of a category y and a bounding box b. The category y is an element of a set Y of categories. In the example of Figure 6, the set Y is "person and bag". From the above, the dataset D with the associated correct labels is the set of image x and all objects contained in image x.

number

number

number

[0055] (Configuration of control unit 100a) As shown in Fig. 5, the control unit 100a includes a learning unit 101, a threshold determination unit 102, an inference unit 103, a dataset generation unit 104, and a re-learning unit 105. As shown in Fig. 5, the threshold determination unit 102 includes an evaluation dataset inference unit 1021, an evaluation value calculation unit 1022, and a threshold judgment unit 1023. As shown in Fig. 5, the dataset generation unit 104 includes a pseudo label generation unit 1041 and an association unit 1042.

[0056] The evaluation dataset inference unit 1021, the evaluation value calculation unit 1022, and the threshold judgment unit 1023 correspond to the threshold determination unit 102 in the exemplary embodiment 1, and are components that realize the threshold determination means in this exemplary embodiment. The pseudo label generation unit 1041 and the association unit 1042 correspond to the dataset generation unit 104 in the exemplary embodiment 1, and are components that realize the dataset generation means in this exemplary embodiment. The re-learning unit 105 is a component that realizes the pseudo label reference learning means in this exemplary embodiment.

[0057] The learning unit 101 acquires a dataset 1 (DS1) and uses the dataset 1 (DS1) to learn an object detection model for generating pseudo labels. That is, the learning unit 101 also functions as an acquisition unit that acquires a first dataset. Specifically, the learning unit 101 reads out the dataset 1 (DS1) stored in the storage unit 150a, and uses the dataset 1 (DS1), that is, a dataset in which a correct answer label is associated with each of one or a plurality of images, to learn an object detection model for generating pseudo labels. Then, the learning unit 101 outputs the learned object detection model for generating pseudo labels to the evaluation dataset inference unit 1021 and the inference unit 103.

[0058] The evaluation dataset inference unit 1021 generates an inference result based on the evaluation dataset. Specifically, the evaluation dataset inference unit 1021 acquires the evaluation dataset DSE and an object detection model for generating pseudo labels, and inputs each of one or more images included in the evaluation dataset DSE to the object detection model for generating pseudo labels to obtain an inference result. More specifically, the evaluation dataset inference unit 1021 reads out the evaluation dataset DSE stored in the storage unit 150a, and inputs it to the object detection model for generating pseudo labels acquired from the learning unit 101. Then, the evaluation dataset inference unit 1021 acquires the inference result output by the object detection model for generating pseudo labels, and outputs the inference result to the evaluation value calculation unit 1022.

[0059] The evaluation dataset DSE is a dataset in which, like dataset 1 (DS1), each object included in each image is associated with a correct answer label. For example, the images included in the evaluation dataset DSE may be a part of the images included in dataset 1 (DS1). Also, for example, the images included in the evaluation dataset DSE may be generated by assigning a correct answer label to a part of the images included in dataset 2 (DS2). Also, for example, the images included in the evaluation dataset DSE may be generated by assigning a correct answer label to an image not included in dataset 1 (DS1) and dataset 2 (DS2).

[0060] The inference result by the evaluation dataset inference unit 1021 is, for each of one or more images included in the evaluation dataset DSE, The location of each of the objects inferred to be objects -Size of each object above - Each of the above object categories The evaluation dataset inference unit 1021 includes data on at least one of the above, and further includes data on the likelihood of inference for each of the above objects. Typically, the inference result by the evaluation dataset inference unit 1021 includes a category, a bounding box, and a confidence level. The confidence level is an example of data on the likelihood of inference, and is, for example, a numerical value with 0 as the minimum value and 1 as the maximum value.

[0061] The evaluation value calculation unit 1022 calculates an evaluation value based on the inference result. Specifically, the evaluation value calculation unit 1022 calculates an evaluation value for each inference result based on a comparison result between each inference result for one or more images included in the evaluation dataset DSE and a correct label for each of the images.

[0062] For example, the evaluation value is the harmonic mean of precision and recall, that is, F-measure. Here, the calculation process of the F-measure executed by the evaluation value calculation unit 1022 will be described.

[0063] Specifically, the evaluation score calculation unit 1022 executes the following processes (1) to (6).

[0064] (1) Sort all inference results in order of reliability.

[0065] (2) Identify an inference result whose reliability is equal to or greater than a reference value. The reference value is, for example, 0.9. As described below, a plurality of F values ​​are calculated in the F value calculation process. The reference value is a different value for each F value. In other words, the above-mentioned value 0.9 can be expressed as the initial value of the reference value.

[0066] (3) Identify whether the identified inference result is TP (True Positive) or FP (false positive). Here, a TP is an inference result in which the degree of overlap between the bounding box and the bounding box of the correct label is equal to or greater than a predetermined value, and the category matches the correct label. Also, an FP is (A) An inference result in which the category matches the correct label, but the degree of overlap between the bounding box and the bounding box of the correct label is below a certain value. (B) Inference results where the bounding boxes overlap and the correct label is in a different category (C) Inference result where there is no correct answer label with overlapping bounding boxes It should be noted that, for example, an Intersection Over Union (IOU) is used as a value indicating the degree of overlap of bounding boxes.

[0067] (4) For the correct labels, identify the correct labels that are false negatives (FNs). FNs are defined as: (D) Correct answer label with no overlapping bounding boxes (E) Inference results with overlapping bounding boxes and correct labels with different categories Either:

[0068] (5) Calculate the precision and recall. The precision is the accuracy rate of the inference result, and is calculated, for example, by precision=number of TPs / (number of TPs+number of FPs). The recall is the proportion of correctly inferred labels among the correct labels, and is calculated, for example, by recall=number of TPs / (number of TPs+number of FNs). Figure 7 is a graph showing the relationship between the precision and recall. As shown in Figure 7, the higher the reliability, the higher the precision, but the lower the recall. On the other hand, the lower the reliability, the lower the precision, but the higher the recall. In this way, there is a trade-off between the precision and the recall. Note that the reliability here is the reference value set in the process of (2).

[0069] (6) Calculate the F-score. The F-score is calculated as (2 × precision × recall) / (precision + recall).

[0070] When the above processes are completed, the evaluation value calculation unit 1022 decreases the reference value and executes processes (2) to (6) again. For example, the evaluation value calculation unit 1022 sets the next reference value to 0.8. In other words, the evaluation value calculation unit 1022 calculates an F value based on the next reference value. The evaluation value calculation unit 1022 repeats processes (2) to (6) to calculate an F value based on each reference value. In this way, a plurality of F values ​​based on different reference values ​​are calculated.

[0071] As an example, the evaluation value calculation unit 1022 repeats the processes (2) to (6) until it calculates an F-value with a reference value equal to or less than the minimum reliability. In this example, the final processes (2) to (6) calculate an F-value for all inference results. Note that, for inference results identified in the second or subsequent processes of (2) that have already been identified in a previous process of (2), the processes (3) and (4) may be omitted, and the results identified in the previous processes of (3) and (4) may be used.

[0072] The evaluation value calculation unit 1022 outputs each calculated evaluation value, that is, F-value, to the threshold value determination unit 1023 in association with the reference value used in the calculation of each F-value.

[0073] In the process of (5), if there are multiple categories in the inference result and the correct label, the evaluation value calculation unit 1022 may calculate the precision and recall for each category. In this example, in the process of (6), the evaluation value calculation unit 1022 calculates an F-score for each category. As a result, each reference value is associated with multiple F-scores calculated for each category.

[0074] Moreover, the evaluation value calculated by the evaluation value calculation unit 1022 is not limited to the F value. For example, the evaluation value may be a value that emphasizes precision or recall. In this example, in the process of (6), the evaluation value calculation unit 1022 calculates, for example, {(1+β 2) × precision × recall} / {(β 2 The evaluation value may be calculated as follows: (β × precision) + recall). β is a value for adjusting the importance of precision relative to recall. If the value of β is in the range of 0<β<1, the evaluation value will emphasize recall, and if the value is in the range of 1<β, the evaluation value will emphasize precision.

[0075] In addition, when calculating a plurality of evaluation values, the method of identifying at least a part of the inference results in the process of (2) is not limited to the above example. For example, the evaluation value calculation unit 1022 may identify a predetermined number of inference results in order of reliability. In this example, the evaluation value calculation unit 1022 increases the predetermined number by a predetermined number each time the process of (6) is completed and the next process of (2) to (6) is started. Then, the evaluation value calculation unit 1022 repeats the processes of (2) to (6) until all the inference results are identified in the process of (2) and the evaluation values ​​are calculated. Note that the increase in the predetermined number in the final process of (2) may be 1 or more and a predetermined number or less. In this example, the calculated evaluation value is linked to the minimum reliability among the reliability of the identified inference results and output to the threshold value determination unit 1023.

[0076] Also, for example, the evaluation value calculation unit 1022 may execute the following process instead of the processes (2) to (6): · Identify TP, FP and FN for every inference result. -Set multiple thresholds for the reliability and identify the number of TPs with reliability above each threshold. · Calculate precision and recall for each number of identified TPs. For each combination of precision and recall calculated, an evaluation value (typical example: F-measure) is calculated. The number of TPs to be identified is proportional to the recall value. In this example, the calculated evaluation values ​​are linked to the threshold values ​​used to identify the number of TPs and are output to the threshold value determination unit 1023.

[0077] The threshold value determination unit 1023 determines the threshold value based on the evaluation value. Specifically, the threshold value determination unit 1023 identifies the maximum value among the acquired multiple F values, and sets the reference value associated with the identified F value as the threshold value. Here, the maximum value among the F values ​​can be expressed as a value that balances the precision and recall. As described above, the F value is calculated by an equation including the precision and recall, so the threshold value determination unit 1023 can be expressed as determining the threshold value by referring to the precision and recall indicated by the comparison result by the evaluation value calculation unit 1022. Also, as described above, the precision and recall are in a trade-off relationship, so the precision and recall at which the F value is maximum are not the points at which the precision or recall is maximum in the graph in FIG. 7, but are, for example, the points indicated by the star in the graph in FIG. 7. The threshold value determination unit 1023 outputs the determined threshold value to the pseudo label generation unit 1041.

[0078] In the case where the F-measure is calculated for each category, the threshold determination unit 1023 sets a threshold for each category. That is, the threshold determination unit 1023 determines a plurality of thresholds for each category, and outputs the plurality of thresholds to the pseudo label generation unit 1041 by linking them with information indicating the corresponding category.

[0079] The inference unit 103 reads out the data set 2 (DS2) stored in the storage unit 150a, inputs each of one or more images included in the data set 2 (DS2) to the object detection model for generating pseudo labels acquired from the learning unit 101, and acquires one or more inference results for each of the images. The inference unit 103 outputs the acquired inference results to the pseudo label generation unit 1041.

[0080] The pseudo label generation unit 1041 generates a pseudo label. Specifically, the pseudo label generation unit 1041 sets, as a pseudo label, an inference result having a reliability equal to or higher than a threshold determined by the threshold determination unit 1023, among one or more inference results by the inference unit 103. The pseudo label generation unit 1041 outputs the inference result set as the pseudo label to the association unit 1042.

[0081] When the pseudo label generation unit 1041 acquires multiple thresholds set for each category, it sets, as a pseudo label, an inference result having a reliability equal to or higher than the threshold set for each category among one or more inference results by the inference unit 103. Specifically, the pseudo label generation unit 1041 classifies the inference results by the inference unit 103 by category, and identifies a corresponding threshold for each classification, in other words, a threshold with which the category matches. Then, for each classification, the pseudo label generation unit 1041 compares the reliability of each inference result with the identified threshold, and sets, as a pseudo label, an inference result having a reliability equal to or higher than the threshold.

[0082] The associating unit 1042 associates the pseudo labels set by the pseudo label generating unit 1041 with the corresponding images. This generates a dataset 2' (DS2') in which the pseudo labels are associated with one or more images included in the dataset 2 (DS2). The associating unit 1042 stores the generated dataset 2' (DS2') in the storage unit 150a and notifies the re-learning unit 105.

[0083] FIG. 8 is a diagram showing a specific example of data included in the data set 2' (DS2'). Specifically, FIG. 8 shows one of the images included in the data set 2' (DS2'). The image is an image included in the data set 2 (DS2) shown in FIG. 6, and a pseudo label is associated with each of five objects included in the image. Typically, the pseudo label is a label including a category and a bounding box as shown in FIG. 8. The category is category information indicating the category of an object included in the image associated with the pseudo label. In the example of FIG. 8, the category of "person" is associated with each of the three people, and the category of "bag" is associated with each of the two bags. The bounding box is area information indicating the area of ​​an object included in the image associated with the pseudo label. One bounding box is associated with one object, and a typical example of the bounding box is data indicating the smallest rectangle in which the object is contained, as shown in FIG. 8.

[0084] The re-learning unit 105 uses the dataset after the pseudo label is added to train the detection model for the target image. As an example, the re-learning unit 105 re-learns the detection model trained by the training unit 101 as the training of the detection model for the target image. Specifically, the re-learning unit 105 reads out the dataset 2' (DS2') from the storage unit 150a, and trains the object detection model DM using the dataset 2' (DS2'). Then, the re-learning unit 105 stores the trained object detection model DM in the storage unit 150a. As another example, the re-learning unit 105 may train a new detection model as the training of the detection model for the target image, and store the new detection model in the storage unit 150a.

[0085] As described above, the information processing device 10a according to this exemplary embodiment employs a configuration in which a target image detection model is trained using a dataset after pseudo-labeling. Therefore, the information processing device 10a according to this exemplary embodiment has the effect of being able to generate a target image detection model while reducing the cost associated with adjusting the threshold. Therefore, the information processing device 10a according to this exemplary embodiment can generate a highly accurate target image detection model while suppressing the generation cost.

[0086] Furthermore, in the information processing device 10a according to this exemplary embodiment, a configuration is adopted in which, as the learning of the detection model for the target image, the detection model learned by the learning unit 101 is re-learned. Therefore, according to the information processing device 10a according to this exemplary embodiment, it is possible to make the detection model more accurate while suppressing the cost of re-learning.

[0087] In addition, the information processing device 10a according to the present exemplary embodiment employs a configuration for automatically determining a threshold for determining whether or not to treat the inference result for each image included in the second data set as a pseudo label. Therefore, the information processing device 10a according to the present exemplary embodiment has an advantage that the number of times re-learning was required each time the threshold was adjusted can be reduced to one. As a result, the time required for re-learning can be reduced, and the time required for generating a detection model can be reduced.

[0088] As described above, the information processing device 10a according to the present exemplary embodiment employs a configuration in which the correct label and the pseudo label include region information and category information. Therefore, the information processing device 10a according to the present exemplary embodiment has an effect of improving the accuracy of detection of an object included in an image using a detection model after re-learning.

[0089] As described above, the information processing device 10a according to the present exemplary embodiment employs a configuration in which a threshold value is determined by referring to the calculated precision rate and recall rate. Therefore, the information processing device 10a according to the present exemplary embodiment has an effect of improving the accuracy of setting pseudo labels. Furthermore, the information processing device 10a according to the present exemplary embodiment can set pseudo labels by taking into account both the quality (precision rate) of the training data and the amount (recall rate) of the training data, and therefore has an effect of generating a highly accurate detection model for target images.

[0090] As described above, the information processing device 10a according to the present exemplary embodiment may be configured to set a threshold value for each category of the inference result of the image included in the evaluation dataset by the object detection model for generating pseudo labels and the ground truth label associated with the image. Therefore, the information processing device 10a according to the present exemplary embodiment that employs such a configuration can improve the accuracy of setting pseudo labels.

[0091] As described above, the information processing device 10a according to the present exemplary embodiment may adopt a configuration in which the images included in the evaluation data set DSE are included in the first data set. Therefore, according to the information processing device 10a according to the present exemplary embodiment adopting this configuration, it is possible to obtain an effect that it is not necessary to newly perform a costly task of correct answer assignment in order to generate the evaluation data set DSE. Furthermore, according to the information processing device 10a according to the present exemplary embodiment adopting this configuration, it is possible to obtain an effect that the number of images to be prepared in advance can be reduced.

[0092] As described above, the information processing device 10a according to the present exemplary embodiment may be configured to generate images included in the evaluation data set DSE by assigning a correct answer label to a part of the second data set. Therefore, according to the information processing device 10a according to the present exemplary embodiment employing this configuration, a part of the data set to which the pseudo label is assigned is used as the evaluation data set DSE to determine the threshold, so that the accuracy of the assigned pseudo label can be improved. Furthermore, according to the information processing device 10a according to the present exemplary embodiment employing this configuration, the number of images to be prepared in advance can be reduced.

[0093] <Information processing method flow> The flow of the information processing method S10a executed by the information processing device 10a configured as above will be described with reference to Fig. 9. Fig. 9 is a flow diagram showing the flow of the information processing method S10a. The information processing device 10a executes the information processing method S10a to generate a second data set including images associated with pseudo labels.

[0094] (Step S101) In step S101, the learning unit 101 learns a detection model. Specifically, the learning unit 101 reads out a dataset 1 (DS1) stored in the storage unit 150a, and learns an object detection model for generating pseudo labels using the dataset 1 (DS1), i.e., a dataset in which one or more images are each associated with a correct label. Then, the learning unit 101 outputs the learned object detection model for generating pseudo labels to the evaluation dataset inference unit 1021 and the inference unit 103.

[0095] (Step S1021) In step S1021, the evaluation dataset inference unit 1021 generates an inference result based on the evaluation dataset. Specifically, the evaluation dataset inference unit 1021 reads out the evaluation dataset DSE stored in the storage unit 150a, and inputs it to the object detection model for generating pseudo labels acquired from the learning unit 101. Then, the evaluation dataset inference unit 1021 acquires the inference result output by the object detection model for generating pseudo labels, and outputs the inference result to the evaluation value calculation unit 1022.

[0096] (Step S1022) In step S1022, the evaluation value calculation unit 1022 calculates an evaluation value based on the inference result. Specifically, the evaluation value calculation unit 1022 calculates a precision and a recall based on a comparison result between an inference result specified based on a reference value among the inference results by the evaluation dataset inference unit 1021 and a correct label for each of one or more images included in the evaluation dataset DSE, and calculates an F value as an evaluation value from the precision and recall. The evaluation value calculation unit 1022 changes the reference value and repeats the calculation of the F value, and calculates multiple F values ​​corresponding to each reference value. The evaluation value calculation unit 1022 links each of the calculated F values ​​to the corresponding reference value and outputs them to the threshold determination unit 1023.

[0097] (Step S1023) In step S1023, the threshold determination unit 1023 determines a threshold based on the evaluation value. Specifically, the threshold determination unit 1023 identifies the maximum value among the acquired multiple F values, and sets the reference value associated with the identified F value as the threshold. The threshold determination unit 1023 outputs the determined threshold to the pseudo label generation unit 1041.

[0098] It should be noted that steps S1021 to S1023 correspond to step S102 described in the first exemplary embodiment.

[0099] (Step S103) In step S103, the inference unit 103 performs inference. Specifically, the inference unit 103 reads out the data set 2 (DS2) stored in the storage unit 150a, inputs one or more images included in the data set 2 (DS2) to the object detection model for generating pseudo labels acquired from the learning unit 101, and acquires one or more inference results for each of the images. The inference unit 103 outputs the acquired inference results to the pseudo label generation unit 1041.

[0100] (Step S1041) In step S1041, the pseudo label generation unit 1041 generates a pseudo label. Specifically, the pseudo label generation unit 1041 sets, as a pseudo label, an inference result having a reliability equal to or higher than the threshold determined by the threshold determination unit 1023, among one or more inference results by the inference unit 103. The pseudo label generation unit 1041 outputs the inference result set as the pseudo label to the association unit 1042.

[0101] (Step S1042) In step S1042, the associating unit 1042 associates images with pseudo labels. Specifically, the associating unit 1042 associates one or more images included in the dataset 2 (DS2) with the corresponding pseudo labels to generate a dataset 2' (DS2'). The pseudo label generating unit 1041 stores the generated dataset 2' (DS2') in the storage unit 150a and notifies the re-learning unit 105.

[0102] It should be noted that steps S1041 to S1042 correspond to step S104 described in the first exemplary embodiment.

[0103] Although not shown in FIG. 9, the re-learning unit 105 uses the dataset after the pseudo label is added to train a detection model for a target image. As an example, the re-learning unit 105 re-learns the detection model trained by the training unit 101 as the training. Specifically, the re-learning unit 105 reads out the dataset 2' (DS2') from the storage unit 150a, and trains the object detection model DM using the dataset 2' (DS2'). Then, the re-learning unit 105 stores the trained object detection model DM in the storage unit 150a. As another example, the re-learning unit 105 may train a new detection model as the training of the detection model for a target image, and store the new detection model in the storage unit 150a.

[0104] As described above, the information processing method S10a according to this exemplary embodiment provides the same effect as the information processing device 10a. That is, the information processing method S10a according to this exemplary embodiment employs a configuration in which a target image detection model is trained using a dataset after pseudo-labeling. Therefore, the information processing method S10a according to this exemplary embodiment provides the effect of reducing the cost of adjusting the threshold and generating a target image detection model used by the information processing device. Therefore, the information processing method S10a according to this exemplary embodiment can generate a highly accurate detection model while suppressing the generation cost.

[0105] <Configuration of information processing device 20a> The configuration of the information processing device 20a according to this exemplary embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the information processing device 20a.

[0106] As shown in Fig. 10, the information processing device 20a includes a control unit 200a, a storage unit 250a, and an output unit 260a. The control unit 200a controls each unit of the information processing device 20a. The storage unit 250a stores various programs and data used by the information processing device 20a. The output unit 260a outputs the information processing result by the information processing device 20a.

[0107] The storage unit 250a stores a target dataset TDS and an object detection model DM. The target dataset TDS is a dataset including one or more target images that are targets for object detection. The object detection model DM is a detection model for a target image, and specifically, the object detection model DM generated by the re-learning unit 105 of the information processing device 10a.

[0108] That is, the object detection model DM is a learning process for learning a detection model using a first data set; A threshold determination process for determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; an inference process for obtaining one or more inference results for each of one or more images included in a second data set by inputting each of the one or more images into the detection model; a dataset generation process for generating a dataset after pseudo-labeling by setting an inference result having a reliability equal to or higher than a threshold value as a pseudo label among one or more inference results by the inference process and associating the pseudo label with a corresponding image; and - Pseudo-label reference learning process that retrains the detection model for the target image by referring to the dataset after pseudo-labeling In other words, the object detection model DM is produced by a method including a step of performing each of the above processes.

[0109] (Configuration of control unit 200a) As shown in FIG. 10, the control unit 200 a includes an acquisition unit 201 and a detection unit 202 .

[0110] The acquisition unit 201 acquires a target image. Specifically, the acquisition unit 201 reads out a target data set TDS from the storage unit 250a and outputs the data set TDS to the detection unit 202.

[0111] The detection unit 202 detects objects included in the target images using a detection model for the target images. Specifically, the detection unit 202 inputs the target images included in the target dataset TDS acquired from the acquisition unit 201 to the object detection model DM, and acquires an inference result output from the object detection model DM. The detection unit 202 outputs the acquired inference result to the output unit 260a. As a result, the output unit 260a outputs the following for each target image: -Whether or not an object is included in the target image - Position of objects in the target image - The size of the object in the target image - The category of objects contained in the target image Typically, the output unit 260a causes a display device to display a target image in which a category and a bounding box are added to at least a part of an object. The display device may be the output unit 260a, or may be a display device (not shown) communicably connected to the information processing device 20a.

[0112] As described above, the information processing device 20a according to this exemplary embodiment employs a configuration in which a pseudo label is determined using an automatically determined threshold, and an object is detected using a target image detection model trained using a dataset including an image associated with the pseudo label. Therefore, the information processing device 20a according to this exemplary embodiment has an effect of being able to detect an object included in an image using a target image detection model that reduces the cost of adjusting the threshold.

[0113] In addition, the information processing device 20a according to the present exemplary embodiment is configured to output an inference result for a target image using a detection model for the target image. Therefore, the information processing device 20a according to the present exemplary embodiment has an effect that a user of the information processing device 20a can recognize the inference result.

[0114] Exemplary embodiment 3 The third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first or second exemplary embodiment are denoted by the same reference numerals, and the description thereof will not be repeated.

[0115] <Configuration of information processing device 10b> The configuration of the information processing device 10b will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the configuration of the information processing device 10b. As shown in Fig. 11, the information processing device 10b includes a control unit 100b and a storage unit 150b. The control unit 100b controls each unit of the information processing device 10b in an integrated manner. The storage unit 150b stores various programs and data used by the information processing device 10b.

[0116] The storage unit 150b differs from the storage unit 150a described in the second exemplary embodiment in the data contained in the data set 2 (DS2). The details of the data will be described with reference to FIG.

[0117] Fig. 12 is a diagram showing specific examples of data included in dataset 1 (DS1) and dataset 2 (DS2). Specifically, Fig. 12 shows one image included in dataset 1 (DS1) and one image included in dataset 2 (DS2).

[0118] In this exemplary embodiment, the images included in the dataset 1 (DS1) include five objects, specifically, three dogs and two cows. The images included in the dataset 2 (DS2) include five objects, specifically, two dogs and three cows. The dataset 1 (DS1) and the dataset 2 (DS2) according to this exemplary embodiment are multiple datasets (also called expert datasets) with different categories (responsibility areas) of correct answers.

[0119] In this exemplary embodiment, unlike the exemplary embodiment 2, at least a part of one or more images included in the dataset 2 (DS2) is assigned with one or more correct labels. In the images included in the dataset 1 (DS1), each of the three dogs is associated with a correct label. That is, the dataset 1 (DS1) is an expert dataset with a responsibility area of ​​"dog". In the images included in the dataset 2 (DS2), each of the three cows including the object Ob1 is associated with a correct label. That is, the dataset 2 (DS2) is an expert dataset with a responsibility area of ​​"cow". Although not shown, the evaluation dataset DSE according to this exemplary embodiment is a dataset including images in which a correct label is associated with a dog, similar to the dataset 1 (DS1).

[0120] (Configuration of control unit 100b) 11, the control unit 100b differs from the control unit 100a described in the second exemplary embodiment in that the control unit 100b includes an associating unit 1042b instead of the associating unit 1042. The pseudo label generating unit 1041 and the associating unit 1042b correspond to the data set generating unit 104 in the first exemplary embodiment, and are configured to realize the data set generating means in this exemplary embodiment.

[0121] The associating unit 1042b has the following function in addition to the functions of the associating unit 1042. That is, when a correct label is assigned to an object included in an image associated with a pseudo label, and the degree of overlap between a region indicated by region information included in the pseudo label and a region indicated by region information included in the correct label is equal to or greater than a predetermined degree, the associating unit 1042b deletes the pseudo label.

[0122] FIG. 13 is a diagram showing specific examples of data included in dataset 2 (DS2), dataset 2' (DS2'), and dataset 2". Specifically, FIG. 13 shows one of the images included in each of dataset 2 (DS2), dataset 2' (DS2'), and dataset 2". Note that dataset 2 (DS2) according to this exemplary embodiment has already been described, so the description will not be repeated here.

[0123] As described in the second exemplary embodiment, the data set 2' (DS2') is a data set in which one or more images included in the data set 2 (DS2) are associated with a pseudo label. The pseudo label can be said to be based on the data set 1 (DS1) and the evaluation data set DSE, so in the example of FIG. 13, a correct label with a category of "dog" is associated with a part of an object. Here, in the example of FIG. 13, a pseudo label including the category of "dog", that is, an incorrect pseudo label, is associated with the object Ob1. Although not shown in FIG. 13, a correct label is associated with the object Ob1 in the images included in the data set 2 (DS2) from which the data set 2' (DS2') is generated, so that the correct label is associated with the object Ob1 in addition to the pseudo label in the images included in the data set 2' (DS2').

[0124] The associating unit 1042b identifies, for each image included in data set 2' (DS2'), a corresponding image from among images included in data set 2 (DS2).

[0125] Next, the associating unit 1042b selects one of the images included in the data set 2' (DS2') and calculates the IOU with the bounding box of the correct label included in the identified image for each of the bounding boxes of the pseudo label associated with the image. The IOU corresponds to the degree of overlap described above. The associating unit 1042b performs this process for all of the images included in the data set 2' (DS2').

[0126] The associating unit 1042b deletes the pseudo label when there is a correct label whose IOU is equal to or greater than a predetermined value. In the example of FIG. 13, the IOU between the pseudo label associated with the object Ob1 and the correct label associated with the object Ob1 is equal to or greater than a predetermined value. Therefore, the associating unit 1042b deletes the pseudo label associated with the object Ob1. The image included in the dataset 2'' shown in FIG. 13 is an image after the deletion of the pseudo label. As shown in FIG. 13, in this image, the pseudo label associated with the object Ob1 has been deleted, and only the correct label is associated with the object Ob1.

[0127] As described above, the information processing device 10b according to the present exemplary embodiment is configured to delete a pseudo label when the degree of overlap between the area indicated by the area information included in the pseudo label and the area indicated by the area information included in the correct label is equal to or greater than a predetermined degree in the pseudo label and the correct label attached to the image. Therefore, according to the information processing device 10b according to the present exemplary embodiment, when a pseudo label is inappropriate, the pseudo label is deleted and the correct label remains, so that it is possible to improve the accuracy of detecting an object using a detection model for a target image. Note that an inappropriate pseudo label refers to, for example, (1) the category of the pseudo label is different from the category of the object, or (2) the bounding box of the pseudo label does not include a part of the object, etc.

[0128] In particular, in the case of an expert data set in which similar-looking objects such as dogs and cows are labeled with correct labels in this exemplary embodiment, there is a high possibility that a pseudo label of an incorrect category will be associated with the object. In contrast, the information processing device 10b according to this exemplary embodiment can delete the incorrect pseudo label, thereby generating a pseudo label with high accuracy, and thus improving the accuracy of object detection using a detection model for a target image.

[0129] Exemplary embodiment 4 A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to third exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

[0130] <Overview of information processing device 10c> The information processing device 10c according to the present exemplary embodiment determines a threshold value based on each of the expert data sets, and assigns a pseudo label to each of the multiple data sets based on the threshold value.

[0131] <Configuration of information processing device 10c> The configuration of the information processing device 10c will be described with reference to FIG. 14. FIG. 14 is a block diagram showing the configuration of the information processing device 10c. As shown in FIG. 14, the information processing device 10c includes a first control unit 100c, a first storage unit 150c, a second control unit 110c, and a second storage unit 160c. The first control unit 100c and the second control unit 110c comprehensively control each unit of the information processing device 10c. The first storage unit 150c and the second storage unit 160c store various programs and data used by the information processing device 10c.

[0132] The first control unit 100c and the second control unit 110c may be integrated. The first storage unit 150c and the second storage unit 160c may be integrated. Alternatively, the second control unit 110c and the second storage unit 160c may be provided in a separate device connected to the information processing device 10c so as to be able to communicate with the information processing device 10c.

[0133] The first storage unit 150c stores a data set 1 (DS1), a data set 2 (DS2), an evaluation data set 1 (DSE1), and an evaluation data set 2 (DSE2). The data set 1 (DS1) is a first data set in this exemplary embodiment. The data set 2 (DS2) is a second data set in this exemplary embodiment. Moreover, the data set 1 (DS1) and the data set 2 (DS2) are the above-mentioned expert data sets. The evaluation data set 1 (DSE1) is a first evaluation data set in this exemplary embodiment. The evaluation data set 2 (DSE2) is a second evaluation data set in this exemplary embodiment.

[0134] Here, the details of dataset 1 (DS1) and dataset 2 (DS2) will be described. Fig. 15 is a diagram showing a specific example of data included in dataset 1 (DS1) and dataset 2 (DS2). Specifically, Fig. 15 shows one of the images included in dataset 1 (DS1) and one of the images included in dataset 2 (DS2).

[0135] Each of these images contains five objects, specifically three people and two bags. In the images contained in Dataset 1 (DS1), each of the two bags has a ground truth label associated with it. That is, Dataset 1 (DS1) is an expert dataset with a responsibility area of ​​"bag". In the images contained in Dataset 2 (DS2), the three people have a ground truth label associated with it. That is, Dataset 2 (DS2) is an expert dataset with a responsibility area of ​​"person".

[0136] The evaluation dataset 1 (DSE1) is a dataset in which a correct answer label is associated with each of the objects in the responsibility range included in each image, similar to the dataset 1 (DS1). Based on the example of FIG. 15, the evaluation dataset 1 (DSE1) is a dataset including images in which a correct answer label is associated with a bag. For example, the images included in the evaluation dataset 1 (DSE1) may be a part of the images included in the dataset 1 (DS1). Also, for example, the images included in the evaluation dataset 1 (DSE1) may be images that are not included in the dataset 1 (DS1) and in which a correct answer label is associated with an object in the responsibility range in the dataset 1 (DS1).

[0137] The evaluation dataset 2 (DSE2) is a dataset in which a correct answer label is associated with each of the objects in the responsibility range included in each image, similar to the dataset 2 (DS2). Based on the example of FIG. 15, the evaluation dataset 2 (DSE2) is a dataset including images in which a correct answer label is associated with a person. For example, the images included in the evaluation dataset 2 (DSE2) may be a part of the images included in the dataset 2 (DS2). Also, for example, the images included in the evaluation dataset 2 (DSE2) may be images that are not included in the dataset 2 (DS2) and in which a correct answer label is associated with an object in the responsibility range in the dataset 2 (DS2).

[0138] (Configuration of the first control unit 100c) As shown in FIG. 14, the first control unit 100c includes a first learning unit 101-1, a second learning unit 101-2, a first threshold determination unit 102-1, a second threshold determination unit 102-2, a first inference unit 103-1, a second inference unit 103-2, a first dataset generation unit 104-1, and a second dataset generation unit 104-2.

[0139] The first learning unit 101-1 is a configuration for realizing the first learning means in this exemplary embodiment. The second learning unit 101-2 is a configuration for realizing the second learning means in this exemplary embodiment. The first threshold value determination unit 102-1 is a configuration for realizing the first threshold value determination means in this exemplary embodiment. The second threshold value determination unit 102-2 is a configuration for realizing the second threshold value determination means in this exemplary embodiment. The first inference unit 103-1 is a configuration for realizing the first inference means in this exemplary embodiment. The second inference unit 103-2 is a configuration for realizing the second inference means in this exemplary embodiment. The first data set generation unit 104-1 is a configuration for realizing the first data set generation means in this exemplary embodiment. The second data set generation unit 104-2 is a configuration for realizing the second data set generation means in this exemplary embodiment.

[0140] The first learning unit 101-1 learns the first detection model using the first data set. Specifically, the first learning unit 101-1 acquires the data set 1 (DS1) and learns the first pseudo label generating object detection model PDM1 using the data set 1 (DS1). More specifically, the first learning unit 101-1 reads out the data set 1 (DS1) stored in the first storage unit 150c and learns the first pseudo label generating object detection model PDM1 using the data set 1 (DS1). Then, the first learning unit 101-1 outputs the learned first pseudo label generating object detection model PDM1 to the first threshold determination unit 102-1 and the first inference unit 103-1.

[0141] The first threshold determination unit 102-1 determines a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the first evaluation dataset into a first detection model and one or more correct labels assigned to each of the one or more images.

[0142] Specifically, the first threshold determination unit 102-1 reads out the evaluation dataset 1 (DSE1) stored in the first storage unit 150c, and inputs it to the first pseudo label generating object detection model PDM1 acquired from the first learning unit 101-1. Then, the first threshold determination unit 102-1 acquires the inference result output by the first pseudo label generating object detection model PDM1.

[0143] Next, the first threshold determination unit 102-1 calculates an evaluation value for each inference result based on a comparison result between each of one or more inference results for each of one or more images included in the evaluation dataset 1 (DSE1) and a correct label for each of the images. The evaluation value is, for example, an F value. Note that, since the details of the calculation process of the F value in the example in which the evaluation value is an F value have been described in the second exemplary embodiment, the description will not be repeated here.

[0144] Next, the first threshold value determination unit 102-1 identifies the maximum value among the multiple F values ​​calculated for each reference value, and sets the reference value associated with the identified F value as a threshold value. This threshold value is the above-mentioned first threshold value. The first threshold value determination unit 102-1 outputs the determined first threshold value to the first data set generation unit 104-1.

[0145] The first inference unit 103-1 inputs each of one or more images included in the second dataset into the first detection model, thereby acquiring one or more inference results for each of the one or more images. Specifically, the first inference unit 103-1 reads out the dataset 2 (DS2) stored in the first storage unit 150c, inputs each of one or more images included in the dataset 2 (DS2) into the first pseudo label generating object detection model PDM1 acquired from the first learning unit 101-1, and acquires one or more inference results PR1 for each of the images. The first inference unit 103-1 outputs the acquired inference results PR1 to the first dataset generation unit 104-1.

[0146] The first dataset generation unit 104-1 sets an inference result having a reliability equal to or greater than a first threshold value among one or more inference results by the first inference unit 103-1 as a pseudo label, and associates the pseudo label with a corresponding image to generate a second dataset after pseudo labeling. Specifically, the first dataset generation unit 104-1 sets an inference result having a reliability equal to or greater than a first threshold value among one or more inference results PR1 by the first inference unit 103-1 as a pseudo label. Then, the first dataset generation unit 104-1 associates the pseudo label with the corresponding image. This generates a dataset 2' (DS2') in which a pseudo label is associated with each of one or more images included in the dataset 2 (DS2). The first dataset generation unit 104-1 stores the generated dataset 2' (DS2') in the second storage unit 160c.

[0147] The second learning unit 101-2 learns the second detection model using the second data set. Specifically, the second learning unit 101-2 acquires the data set 2 (DS2) and learns the second pseudo label generation object detection model PDM2 using the data set 2 (DS2). More specifically, the second learning unit 101-2 reads out the data set 2 (DS2) stored in the first storage unit 150c and learns the second pseudo label generation object detection model PDM2 using the data set 2 (DS2). Then, the second learning unit 101-2 outputs the learned second pseudo label generation object detection model PDM2 to the second threshold determination unit 102-2 and the second inference unit 103-2.

[0148] The second threshold determination unit 102-2 determines the second threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the second evaluation dataset into a second detection model and one or more correct labels assigned to each of the one or more images.

[0149] Specifically, the second threshold determination unit 102-2 reads out the evaluation data set 2 (DSE2) stored in the first storage unit 150c, and inputs it to the second pseudo label generating object detection model PDM2 acquired from the second learning unit 101-2. Then, the second threshold determination unit 102-2 acquires the inference result output by the second pseudo label generating object detection model PDM2.

[0150] Next, the second threshold determination unit 102-2 calculates an evaluation value for each inference result based on a comparison result between each inference result for one or more images included in the evaluation dataset 2 (DSE2) and a correct label for each of the images. The evaluation value is, for example, an F value. Note that the details of the calculation process of the F value in the example in which the evaluation value is an F value have been described in the second exemplary embodiment, and therefore will not be repeated here.

[0151] Next, the second threshold determination unit 102-2 identifies the maximum value among the multiple F values ​​calculated for each reference value, and sets the reference value associated with the identified F value as the threshold. This threshold is the above-mentioned second threshold. The second threshold determination unit 102-2 outputs the determined second threshold to the second data set generation unit 104-2.

[0152] The second inference unit 103-2 inputs each of one or more images included in the first dataset into a second detection model, thereby acquiring one or more inference results for each of the one or more images. Specifically, the second inference unit 103-2 reads out the dataset 1 (DS1) stored in the first storage unit 150c, inputs each of one or more images included in the dataset 1 (DS1) into the second pseudo label generation object detection model PDM2 acquired from the second learning unit 101-2, and acquires one or more inference results PR2 for each of the images. The second inference unit 103-2 outputs the acquired inference results PR2 to the second dataset generation unit 104-2.

[0153] The second dataset generation unit 104-2 sets an inference result having a reliability equal to or greater than a second threshold value among one or more inference results by the second inference unit 103-2 as a pseudo label, and associates the pseudo label with a corresponding image to generate a first dataset after pseudo labeling. Specifically, the second dataset generation unit 104-2 sets an inference result having a reliability equal to or greater than a second threshold value among one or more inference results PR2 by the second inference unit 103-2 as a pseudo label. Then, the second dataset generation unit 104-2 associates the pseudo label with a corresponding image. This generates a dataset 1' (DS1') in which a pseudo label is associated with each of one or more images included in the dataset 1 (DS1). The second dataset generation unit 104-2 stores the generated dataset 1' (DS1') in the second storage unit 160c.

[0154] Here, data set 1' (DS1') and data set 2' (DS2') will be described in detail. Fig. 16 is a diagram showing a specific example of data included in data set 1' (DS1') and data set 2' (DS2'). Specifically, Fig. 16 shows one of the images included in data set 1' (DS1') and one of the images included in data set 2' (DS2').

[0155] The images included in the dataset 1' (DS1') shown in FIG. 16 are the same as the images included in the dataset 1 (DS1) (see FIG. 15). For the objects in the images included in the dataset 1' (DS1'), two bags are associated with a correct label, and three people are associated with a pseudo label. The correct label is the correct label associated with the bag, which is the responsibility area of ​​the dataset 1 (DS1), in the image included in the dataset 1 (DS1) from which the dataset 1' (DS1') was generated. The pseudo label is a pseudo label set by the second dataset generation unit 104-2 based on the inference result PR2. The inference result PR2 is an inference result using the second pseudo label generating object detection model PDM2, which has been trained using the dataset 2 (DS2) whose responsibility area is a person, and therefore the pseudo label is associated with a person.

[0156] The images included in the dataset 2' (DS2') shown in FIG. 16 are the same as the images included in the dataset 2 (DS2) (see FIG. 15). For the objects of the images included in the dataset 2' (DS2'), each of the three people is associated with a correct label, and each of the two bags is associated with a pseudo label. The correct labels are correct labels associated with the people who are in the responsibility range of the dataset 2 (DS2) in the images included in the dataset 2 (DS2) from which the dataset 2' (DS2') was generated. The pseudo labels are pseudo labels set by the first dataset generation unit 104-1 based on the inference result PR1. The inference result PR1 is an inference result using the first pseudo label generating object detection model PDM1 that has been trained using the dataset 1 (DS1) whose responsibility range is the bag, and therefore the pseudo label is associated with the bag.

[0157] The second storage unit 160c stores a data set 1' (DS1'), a data set 2' (DS2'), and an object detection model DM. The data set 1' (DS1') and the data set 2' (DS2') are data sets generated by the second data set generation unit 104-2 and the first data set generation unit 104-1, respectively. The object detection model DM is a detection model for a target image, and will be described in detail later.

[0158] (Configuration of the second control unit 110c) As shown in FIG. 14, the second control unit 110c includes a re-learning unit 105. The re-learning unit 105 is configured to realize a pseudo label reference learning means in this exemplary embodiment. The re-learning unit 105 uses the dataset after the pseudo label is added to learn a detection model for a target image. Specifically, the re-learning unit 105 re-learns the first pseudo label generating object detection model PDM1 or the second pseudo label generating object detection model PDM2 as the learning. More specifically, the re-learning unit 105 reads out the dataset 1' (DS1') and the dataset 2' (DS2') from the second storage unit 160c, and re-learns the first pseudo label generating object detection model PDM1 or the second pseudo label generating object detection model PDM2 using the dataset 1' (DS1') and the dataset 2' (DS2'). Then, the re-learning unit 105 stores the object detection model DM generated by the re-learning in the second storage unit 160c. The re-learning unit 105 may use the data set 1' (DS1') and the data set 2' (DS2') to train a new object detection model DM. The new object detection model DM is a detection model for a target image different from both the first object detection model PDM1 for generating pseudo label and the second object detection model PDM2 for generating pseudo label.

[0159] As described above, the information processing device 10c according to the present exemplary embodiment adopts a configuration in which a threshold is determined based on each of a plurality of expert data sets, and a pseudo label is assigned to each of the plurality of data sets based on the threshold. Therefore, according to the information processing device 10c according to the present exemplary embodiment, a detection model can be retrained using a plurality of data sets to which a pseudo label is assigned, specifically, data set 1' (DS1') and data set 2' (DS2'), so that the detection accuracy of an object included in an image using the retrained detection model can be further improved. In addition, according to the information processing device 10c according to the present exemplary embodiment, even when a plurality of data sets to which a pseudo label is assigned is generated, that is, even when a plurality of thresholds for determining a pseudo label are required, the plurality of thresholds can be automatically determined, so that the cost related to threshold adjustment can be reduced. In addition, according to the information processing device 10c according to the present exemplary embodiment, an effect is obtained in which a single highly accurate detection model for a target image can be learned from a plurality of data sets with different responsibility ranges.

[0160] In this exemplary embodiment, an example in which the number of expert data sets is "2" has been described, but the number of expert data sets is not limited to this example. The number of data sets and evaluation data sets stored in the information processing device 10c, and the number of components that realize the learning means, threshold value determination means, inference means, and data set generation means in the information processing device 10c, correspond to the number of expert data sets. For example, when the number of expert data sets is "3", the information processing device 10c further stores a third data set and a third evaluation data set, and further includes a third learning unit, a third threshold value determination unit, a third inference unit, and a third data set generation unit.

[0161] Furthermore, in this exemplary embodiment, the expert data sets have been described as having different areas of responsibility, but the areas of responsibility may overlap between the expert data sets.

[0162] Furthermore, the first dataset generation unit 104-1 and the second dataset generation unit 104-2 according to the present exemplary embodiment may have the function of the association unit 1042b described in the exemplary embodiment 3. That is, the first dataset generation unit 104-1 may delete the pseudo label when a correct label is assigned to an object included in an image associated with a pseudo label for the dataset 2' (DS2') and the degree of overlap between the area indicated by the area information included in the pseudo label and the area indicated by the area information included in the correct label is equal to or greater than a predetermined degree. The second dataset generation unit 104-2 may delete the pseudo label when a correct label is assigned to an object included in an image associated with a pseudo label for the dataset 1' (DS1)' and the degree of overlap between the area indicated by the area information included in the pseudo label and the area indicated by the area information included in the correct label is equal to or greater than a predetermined degree.

[0163] Exemplary embodiment 5 The fifth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to fourth exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

[0164] <Configuration of information processing device 10d> The configuration of an information processing device 10d according to this exemplary embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing the configuration of the information processing device 10d. As shown in Fig. 17, the information processing device 10d includes a control unit 100d and a storage unit 150d. The control unit 100d controls each unit of the information processing device 10d. The storage unit 150d stores various programs and data used by the information processing device 10d.

[0165] The control unit 100d includes a non-learning region determination unit 106 in addition to the learning unit 101, threshold determination unit 102, inference unit 103, data set generation unit 104, and re-learning unit 105 according to the above-described exemplary embodiment 2. The non-learning region determination unit 106 is configured to realize a non-learning region determination means in this exemplary embodiment.

[0166] The threshold determination unit 102 determines the first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset DSE to a detection model and one or more correct labels attached to each of the one or more images, as in the above-mentioned exemplary embodiment 2. The method by which the threshold determination unit 102 determines the first threshold has already been described in the above-mentioned exemplary embodiment 2, and therefore will not be repeated here.

[0167] In addition, in this exemplary embodiment, the threshold determination unit 102 further determines a second threshold value smaller than the first threshold value by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset DSE into the detection model and one or more correct labels assigned to each of the one or more images.

[0168] The second threshold is a value smaller than the first threshold. For example, the first threshold may be a value that emphasizes precision, and the second threshold may be a value that emphasizes recall. For example, the first threshold may be F 0.5 The second threshold is the confidence level that prioritizes recall. 2 The reliability may be the reliability at which the -score has the maximum value.

[0169] The non-learning area determination unit 106 determines an area corresponding to one or more inference results by the inference unit 103 in the pseudo-labeled dataset 2' (DS2') generated by the dataset generation unit 104, which has a reliability lower than the first threshold and higher than or equal to the second threshold, as a non-learning area that is not to be learned by the re-learning unit 105.

[0170] <Information processing method flow> The flow of the information processing method S10d executed by the information processing device 10d configured as above will be described with reference to Fig. 18. Fig. 18 is a flow diagram showing the flow of the information processing method S10d. The information processing method S10d includes steps S101 to S1022, S1023d, S103 to S1041, S1041d, and S1042. Of these steps, steps S101 to S1022, S103 to S1041, and S1042 have already been described in the above-mentioned exemplary embodiment 2, so the description will not be repeated here.

[0171] (Step S1023d) In step S1023d, the threshold determination unit 1023 determines a first threshold and a second threshold based on the evaluation value. Specifically, the threshold determination unit 1023 identifies the maximum value among the acquired multiple F values, for example, those that emphasize precision, and sets a reference value associated with the identified F value as the first threshold. The threshold determination unit 1023 also identifies the maximum value among the acquired multiple F values, for example, those that emphasize recall, and sets a reference value associated with the identified F value as the second threshold. The threshold determination unit 1023 outputs the determined first threshold and second threshold to the pseudo label generation unit 1041.

[0172] (Step S1041d) In step S1041d, the non-learning area determination unit 106 determines an area corresponding to one or more inference results by the inference unit 103 in the pseudo-labeled dataset 2' (DS2') generated by the dataset generation unit 104, which has a reliability lower than the first threshold and higher than or equal to the second threshold, as a non-learning area that is not to be learned by the re-learning unit 105.

[0173] As described above, in the information processing device 10d according to the present exemplary embodiment, in the dataset after the pseudo label is added generated by the dataset generating unit 104, among one or more inference results by the inference unit 103, an area corresponding to an inference result having a reliability less than the first threshold and equal to or greater than the second threshold is determined as a non-learning area that is not to be learned by the re-learning unit 105. Even if a pseudo label is added to an area corresponding to an inference result having a reliability less than the first threshold and equal to or greater than the second threshold, the pseudo label tends to be low in reliability. By setting such an area as a non-learning area, re-learning can be performed using a pseudo label with a relatively high reliability, so that the detection accuracy of the target image detection model (detection model learned by the learning unit 101) by the re-learning unit 105 can be improved.

[0174] In addition, according to the information processing device 10d employing this configuration, the detection accuracy of the detection model for the target image can be improved, thereby making it possible to reduce the cost of generating a dataset including images to which pseudo labels are assigned using a first threshold and a second threshold.

[0175] Exemplary Embodiment 6 A sixth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to fifth exemplary embodiments are denoted by the same reference numerals, and the description thereof will not be repeated.

[0176] <Configuration of information processing device 10e> The configuration of an information processing device 10e according to this exemplary embodiment will be described with reference to FIG. 19. FIG. 19 is a block diagram showing the configuration of an information processing device 10d. As shown in FIG. 19, the information processing device 10e includes a first control unit 100e, a second control unit 110e, a first storage unit 150e, and a second storage unit 160e. The first control unit 100e and the second control unit 110e collectively control each unit of the information processing device 10e. The first storage unit 150e and the second storage unit 160e store various programs and data used by the information processing device 10e.

[0177] The first control unit 100e includes a first non-learning region determining unit 106-1 and a second non-learning region determining unit 106-2 in addition to the configuration of the first control unit 100c of the information processing device 10c shown in the above-mentioned exemplary embodiment 4. The first non-learning region determining unit 106-1 is configured to realize the first non-learning region determining means in this exemplary embodiment. The second non-learning region determining unit 106-2 is configured to realize the second non-learning region determining means in this exemplary embodiment.

[0178] As in the above-described exemplary embodiment 4, the first threshold determination unit 102-1 determines the first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset 1 (DSE1) into a detection model and one or more correct labels attached to each of the one or more images. The method by which the first threshold determination unit 102-1 determines the first threshold has already been described in the above-described exemplary embodiment 4, and therefore will not be repeated here.

[0179] In addition, in this exemplary embodiment, the first threshold determination unit 102-1 further determines a third threshold smaller than the first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in evaluation dataset 1 (DSE1) into a detection model and one or more correct labels attached to each of the one or more images.

[0180] The third threshold is a value smaller than the first threshold. For example, the first threshold may be a value that emphasizes precision, and the third threshold may be a value that emphasizes recall. For example, the first threshold may be F 0.5 The third threshold is the confidence level that prioritizes recall. 2 The reliability may be the reliability at which the -score has the maximum value.

[0181] The second threshold determination unit 102-2 determines the second threshold by referring to a comparison result between one or more inference results obtained by inputting one or more images included in the evaluation dataset 2 (DSE2) into a detection model and one or more correct labels attached to the one or more images, as in the above-mentioned exemplary embodiment 4. The method by which the second threshold determination unit 102-2 determines the second threshold has already been described in the above-mentioned exemplary embodiment 4, and therefore will not be described again here.

[0182] In addition, in this exemplary embodiment, the second threshold determination unit 102-2 further determines a fourth threshold smaller than the second threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in evaluation dataset 2 (DSE2) into the detection model and one or more correct labels attached to each of the one or more images.

[0183] The fourth threshold is a value smaller than the second threshold. For example, the second threshold may be a value that emphasizes precision, and the fourth threshold may be a value that emphasizes recall. For example, the second threshold may be F 0.5 The fourth threshold is the confidence level that prioritizes recall. 2 The reliability may be the reliability at which the -score has the maximum value.

[0184] The first non-learning area determination unit 106-1 determines an area corresponding to one or more inference results by the first inference unit 103-1 in the pseudo-labeled dataset 2' (DS2') generated by the first dataset generation unit 104-1, which has a reliability lower than the first threshold and higher than or equal to the third threshold, as a non-learning area that is not to be learned by the re-learning unit 105.

[0185] The second non-learning area determination unit 106-2 determines an area corresponding to one or more inference results by the second inference unit 103-2 in the pseudo-labeled dataset 1' (DS1') generated by the second dataset generation unit 104-2, which has a reliability lower than the second threshold and higher than or equal to the fourth threshold, as a non-learning area that is not to be learned by the re-learning unit 105.

[0186] As described above, in the information processing device 10e according to the present exemplary embodiment, in the second data set 2' (DS2') after the pseudo labeling generated by the first data set generating unit 104-1, among one or more inference results by the first inference unit 103-1, a region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the third threshold is determined as a non-learning region that is not to be learned by the re-learning unit 105. Also, in the information processing device 10e, in the first data set 1' (DS1') after the pseudo labeling generated by the second data set generating unit 104-2, among one or more inference results by the second inference unit 103-2, a region corresponding to an inference result having a reliability lower than the second threshold and equal to or higher than the fourth threshold is determined as a non-learning region that is not to be learned by the re-learning unit 105.

[0187] A region corresponding to an inference result having a reliability level less than the first threshold and equal to or greater than the third threshold tends to have a low-reliability pseudo label even if a pseudo label is assigned to the region. A region corresponding to an inference result having a reliability level less than the second threshold and equal to or greater than the fourth threshold tends to have a low-reliability pseudo label even if a pseudo label is assigned to the region. By setting such a region as a non-learning region, re-learning can be performed using a pseudo label having a relatively high reliability, and therefore the detection accuracy of the target image detection model (detection model trained by the first learning unit 101-1 and the second learning unit 101-2) by the re-learning unit 105 can be improved.

[0188] Furthermore, according to the information processing device 10e employing this configuration, the detection accuracy of the detection model for the target image can be improved, thereby making it possible to reduce the cost of generating a dataset including images to which pseudo labels are assigned using the first threshold, the second threshold, the third threshold, and the fourth threshold.

[0189] [Software implementation example] Some or all of the functions of the information processing devices 10, 10a to 10e, 20, and 20a may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0190] In the latter case, the information processing devices 10, 10a to 10e, 20, and 20a are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 20. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing devices 10, 10a to 10e, 20, and 20a. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 10, 10a to 10e, 20, and 20a.

[0191] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0192] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0193] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0194] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0195] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the aspects described below.

[0196] (Appendix 1) A learning means for learning a detection model using a first data set; a threshold determination means for determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; inference means for inputting each of one or more images included in a second data set into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating means for generating a dataset after the pseudo labels are assigned by setting an inference result having a reliability equal to or higher than the first threshold value among one or more inference results by the inference means as a pseudo label and associating the pseudo label with a corresponding image; An information processing device comprising:

[0197] According to the configuration of Supplementary Note 1, a first threshold for setting a pseudo label is automatically determined based on a comparison between an inference result of an image included in an evaluation dataset by a detection model trained using a first dataset and a ground truth label associated with the image. Therefore, according to the configuration of Supplementary Note 1, it is possible to reduce the cost related to adjustment of the first threshold. And, according to the configuration of Supplementary Note 1, from the inference result of an image included in a second dataset by the detection model, an inference result having a reliability equal to or higher than the automatically determined first threshold is set as a pseudo label, and the pseudo label is associated with the corresponding image. Therefore, according to the configuration of Supplementary Note 1, it is possible to reduce the cost of generating a dataset including images to which pseudo labels are assigned.

[0198] (Appendix 2) 2. The information processing device according to claim 1, The system further includes a pseudo-label reference learning means for learning a target image detection model for detecting an object included in the target image by using the pseudo-labeled dataset. 23. An information processing apparatus comprising:

[0199] According to the configuration of Supplementary Note 2, a detection model for a target image is trained using a dataset after pseudo-labeling. Therefore, according to the configuration of Supplementary Note 2, it is possible to generate a detection model for a target image by reducing the cost related to adjusting the threshold. As a result, the cost required for training the detection model for a target image can be reduced. Furthermore, if an appropriate value can be determined as the threshold, the number of times the threshold is adjusted can be reduced, and the number of times the detection model for a target image is trained (retrained) that is required each time the threshold is adjusted can be reduced. As a result, the time required to complete training of the detection model for a target image can be reduced.

[0200] (Appendix 3) The information processing device according to claim 2, The pseudo-label reference learning means re-learns the detection model as the learning of the detection model for the target image. 23. An information processing apparatus comprising:

[0201] According to the configuration of Supplementary Note 3, the detection model is retrained using the dataset after the pseudo-label is added. Therefore, according to the configuration of Supplementary Note 3, the cost required for retraining the detection model can be reduced. Furthermore, if an appropriate value can be determined as the threshold, the number of times the threshold is adjusted can be reduced, and the number of times the detection model is retrained, which is required each time the threshold is adjusted, can be reduced. As a result, the time required to complete retraining the detection model can be reduced.

[0202] (Appendix 4) The information processing device according to claim 2 or 3, The threshold value determining means determining a second threshold value that is smaller than the first threshold value by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels assigned to each of the one or more images; The information processing device includes: The method further includes a non-learning region determining means for determining, in the dataset after the pseudo labels are assigned and generated by the dataset generating means, a region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold, among one or more inference results by the inference means, as a non-learning region that is not to be learned by the pseudo label reference learning means. 23. An information processing apparatus comprising:

[0203] A region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold tends to have a low-reliability pseudo label even if the region is assigned a pseudo label. By setting such a region as a non-learning region, re-learning can be performed using a pseudo label having a relatively high reliability. Therefore, according to the configuration of Supplementary Note 4, the detection accuracy of the target image detection model by the pseudo label reference learning means can be improved.

[0204] (Appendix 5) An information processing device according to any one of claims 1 to 4, The correct label includes area information indicating an area of ​​an object included in an image associated with the correct label, and category information indicating a category of the object, The pseudo label includes area information indicating an area of ​​an object included in an image associated with the pseudo label, and category information indicating a category of the object. 23. An information processing apparatus comprising:

[0205] According to the configuration of Supplementary Note 5, the correct label and the pseudo label include region information and category information. Therefore, according to the configuration of Supplementary Note 5, it is possible to improve the accuracy of detecting objects included in an image using a detection model that has been retrained using a dataset after pseudo labels have been added.

[0206] (Appendix 6) The information processing device according to claim 5, At least a portion of the one or more images included in the second dataset are labeled with one or more correct answer labels; The data set generation means When a correct label is assigned to an object included in an image associated with the pseudo label, and the degree of overlap between a region indicated by region information included in the pseudo label and a region indicated by region information included in the correct label is equal to or greater than a predetermined degree, the pseudo label is deleted. 23. An information processing apparatus comprising:

[0207] According to the configuration of Supplementary Note 6, when the degree of overlap between the area indicated by the area information included in the pseudo label and the area indicated by the area information included in the correct label in the pseudo label and the correct label attached to the image included in the second data set is equal to or greater than a predetermined degree, the pseudo label is deleted. Therefore, according to the configuration of Supplementary Note 6, when the pseudo label is inappropriate, the pseudo label is deleted and the correct label remains, so that it is possible to improve the accuracy of object detection using the detection model after re-learning. In particular, in the case of a data set in which correct labels are attached to objects that look similar, there is a high possibility that a pseudo label of an incorrect category is associated with the object. In contrast, according to the configuration of Supplementary Note 6, since the incorrect pseudo label can be deleted, a pseudo label can be generated with high accuracy, and it is possible to improve the accuracy of object detection using the detection model for the target image.

[0208] Note that an inappropriate pseudo label refers to, for example, (1) the category of the pseudo label is different from the category of the object, or (2) the bounding box of the pseudo label does not include part of the object.

[0209] (Appendix 7) 7. The information processing device according to claim 5, The threshold value determination means sets the first threshold value for each category, The data set generation means Among one or more inference results by the inference means, an inference result having a reliability equal to or higher than the first threshold set for each category is set as a pseudo label, and the pseudo label is associated with a corresponding image, thereby generating a pseudo-labeled dataset. 23. An information processing apparatus comprising:

[0210] According to the configuration of Supplementary Note 7, a first threshold is set for each category of an inference result of an image included in an evaluation dataset by a detection model trained using a first dataset and a ground truth label associated with the image, and an inference result equal to or greater than the first threshold is set as a pseudo label. Therefore, according to the configuration of Supplementary Note 7, it is possible to improve the accuracy of setting the pseudo label.

[0211] (Appendix 8) 8. An information processing device according to any one of claims 1 to 7, The threshold value determination means determines the first threshold value by referring to the precision rate and recall rate indicated by the comparison result. 23. An information processing apparatus comprising:

[0212] According to the configuration of Supplementary Note 8, the first threshold is determined by referring to the precision and recall calculated from the comparison result between the inference result of the image included in the evaluation dataset by the detection model trained using the first dataset and the correct label associated with the image. Therefore, according to the configuration of Supplementary Note 8, it is possible to improve the accuracy of setting the pseudo label. Furthermore, according to the configuration of Supplementary Note 8, it is possible to set the pseudo label taking into consideration both the quality of the training data (precision) and the amount of the training data (recall), so that it is possible to generate a highly accurate detection model for the target image.

[0213] (Appendix 9) An information processing device according to any one of claims 1 to 8, The evaluation data set is included in the first data set. 23. An information processing apparatus comprising:

[0214] According to the configuration of Supplementary Note 9, the images included in the evaluation dataset are included in the first dataset. Therefore, according to the configuration of Supplementary Note 9, it is not necessary to newly perform a costly task of correct answer assignment in order to generate the evaluation dataset. In addition, according to the configuration of Supplementary Note 9, it is possible to reduce the number of images to be prepared in advance.

[0215] (Appendix 10) An information processing device according to any one of claims 1 to 8, The evaluation dataset is generated by adding a correct answer label to a part of the second dataset. 23. An information processing apparatus comprising:

[0216] According to the configuration of Supplementary Note 10, the images included in the evaluation dataset are generated by assigning a correct answer label to a part of the second dataset. Therefore, according to the configuration of Supplementary Note 10, a part of the dataset to which the pseudo label is assigned is used as the evaluation dataset to determine the threshold, so that it is possible to improve the accuracy of the assigned pseudo label. In addition, according to the configuration of Supplementary Note 10, it is possible to reduce the number of images to be prepared in advance.

[0217] (Appendix 11) a first learning means for learning a first detection model using a first data set; A second learning means for learning a second detection model using a second dataset; a first threshold determination means for determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a first evaluation dataset into the first detection model and one or more correct labels attached to each of the one or more images; a second threshold determination means for determining a second threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a second evaluation dataset into the second detection model and one or more correct labels attached to each of the one or more images; a first inference means for inputting each of one or more images included in the second dataset into the first detection model to obtain one or more inference results for each of the one or more images; a second inference means for inputting each of one or more images included in the first dataset into the second detection model to obtain one or more inference results for each of the one or more images; a first dataset generation means for generating a second dataset after assigning a pseudo label to an inference result having a reliability equal to or higher than the first threshold value among one or more inference results by the first inference means, and associating the pseudo label with a corresponding image; a second dataset generation means for generating a first dataset after assigning a pseudo label to an inference result having a reliability equal to or higher than the second threshold value among one or more inference results by the second inference means, and associating the pseudo label with a corresponding image; An information processing device comprising:

[0218] According to the configuration of Supplementary Note 11, a first threshold for setting a pseudo label to a second dataset is automatically determined based on a comparison between an inference result of an image included in a first evaluation dataset by a detection model trained using a first dataset and a correct answer label associated with the image. Also, according to the description of Supplementary Note 11, a second threshold for setting a pseudo label to a first dataset is automatically determined based on a comparison between an inference result of an image included in a second evaluation dataset by a detection model trained using a second dataset and a correct answer label associated with the image. Therefore, according to the configuration of Supplementary Note 11, it is possible to reduce the cost related to the adjustment of the first threshold and the second threshold. That is, according to the configuration of Supplementary Note 11, even when two datasets each having a pseudo label assigned thereto are generated, it is possible to reduce the cost related to the adjustment of two thresholds for setting a pseudo label to each of these two datasets. And, according to the configuration of Supplementary Note 11, re-learning of the detection model is performed using two datasets to which pseudo labels are assigned, so it is possible to further improve the detection accuracy of an object included in an image using the re-learned detection model.

[0219] (Appendix 12) 12. The information processing device according to claim 11, The method further includes a pseudo-label reference learning means for learning a target image detection model for detecting an object included in the target image using the first dataset after the pseudo-label is added and the second dataset after the pseudo-label is added. 23. An information processing apparatus comprising:

[0220] According to the configuration of Supplementary Note 12, the first dataset after the pseudo label is added and the second dataset after the pseudo label is added are used to learn the detection model for the target image. Therefore, according to the configuration of Supplementary Note 12, it is possible to generate the detection model for the target image by reducing the cost related to the adjustment of the first threshold and the adjustment of the second threshold. As a result, the cost until the learning of the detection model for the target image can be reduced. Furthermore, if appropriate values ​​can be determined as the first threshold and the second threshold, the number of times the threshold is adjusted can be reduced, and the number of times the detection model for the target image is learned (re-learned) that is required each time the threshold is adjusted can be reduced. As a result, the time until the learning of the detection model for the target image is completed can be reduced.

[0221] (Appendix 13) 13. The information processing device according to claim 12, The pseudo-label reference learning means re-learns the first detection model and the second detection model as the learning of the detection model for the target image. 23. An information processing apparatus comprising:

[0222] According to the configuration of Supplementary Note 13, the first detection model and the second detection model are retrained using the first dataset after the pseudo label is added and the second dataset after the pseudo label is added. Therefore, according to the configuration of Supplementary Note 13, the cost of retraining the first detection model and the second detection model can be reduced. In addition, if appropriate values ​​can be determined as the first threshold and the second threshold, the number of times the threshold is adjusted can be reduced, and the number of times the detection model is retrained, which is required every time the threshold is adjusted, can be reduced. As a result, the time required to complete retraining the detection model can be reduced.

[0223] (Appendix 14) 14. The information processing device according to claim 12, The first threshold value determining means determining a third threshold value that is smaller than the first threshold value by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a first evaluation dataset into the first detection model and one or more correct labels assigned to each of the one or more images; The second threshold value determining means determining a fourth threshold value that is smaller than the second threshold value by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a second evaluation dataset into the second detection model and one or more correct labels assigned to each of the one or more images; The information processing device includes: a first non-learning region determining means for determining, in the second dataset after the pseudo labels are assigned and generated by the first dataset generating means, a region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the third threshold among one or more inference results by the first inference means, as a non-learning region that is not to be learned by the pseudo label reference learning means; a second non-learning region determining means for determining, in the first dataset after the pseudo labels are assigned and generated by the second dataset generating means, a region corresponding to an inference result having a reliability lower than the second threshold and equal to or higher than the fourth threshold, among one or more inference results by the second inference means, as a non-learning region that is not to be learned by the pseudo label reference learning means; An information processing device comprising:

[0224] A region corresponding to an inference result having a reliability level less than the first threshold and equal to or greater than the third threshold tends to have a low-reliability pseudo label even if a pseudo label is assigned to the region. Also, a region corresponding to an inference result having a reliability level less than the second threshold and equal to or greater than the fourth threshold tends to have a low-reliability pseudo label even if a pseudo label is assigned to the region. By setting such a region as a non-learning region, according to the configuration of appendix 14, re-learning can be performed using a pseudo label having a relatively high reliability, and therefore the detection accuracy of the target image detection model by the pseudo label reference learning means can be improved.

[0225] (Appendix 15) An acquisition means for acquiring a target image; A detection means for detecting an object included in a target image by using a detection model for the target image; Equipped with The detection model for the target image is A learning process for learning a detection model using a first data set; a threshold determination process for determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct answer labels attached to each of the one or more images; an inference process for obtaining one or more inference results for each of one or more images included in a second data set by inputting each of the one or more images into the detection model; a data set generation process for generating a data set after a pseudo label is assigned by setting an inference result having a reliability equal to or higher than the threshold value among one or more inference results by the inference process as a pseudo label and associating the pseudo label with a corresponding image; and A pseudo-label reference learning process for learning the detection model for the target image by referring to the dataset after the pseudo labels are added. It was learned by 23. An information processing apparatus comprising:

[0226] According to the configuration of Supplementary Note 15, a pseudo label is determined using an automatically determined threshold, and an object included in a target image is detected using a detection model for a target image trained using a dataset including an image associated with the pseudo label. Therefore, according to the configuration of Supplementary Note 15, it is possible to detect an object included in a target image using a detection model for a target image with reduced cost for adjusting a threshold.

[0227] (Appendix 16) 16. The information processing device according to claim 15, In the threshold value determination process, a second threshold value smaller than the first threshold value is also determined by referring to the comparison result; In the data set generation process, in the data set after the pseudo labels are assigned, a region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold among one or more inference results by the inference process is determined as a non-learning region that is not to be learned by the pseudo label reference learning process; In the pseudo-label reference learning process, the detection model for the target image is learned by referring to the pseudo-labeled data set including the non-learning region. 23. An information processing apparatus comprising:

[0228] A region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold tends to have a low-reliability pseudo label even if the region is assigned a pseudo label. By setting such a region as a non-learning region, re-learning can be performed using a pseudo label with a relatively high reliability, so that the configuration of appendix 16 can improve the detection accuracy of the target image detection model in the pseudo label reference learning process.

[0229] (Appendix 17) A learning process for learning a detection model using a first dataset; a threshold determination step of determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; an inference step of inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating step of setting an inference result having a reliability equal to or higher than the threshold value as a pseudo label among one or more inference results obtained by the inference step, and associating the pseudo label with a corresponding image, thereby generating a dataset after the pseudo label is assigned; An information processing method comprising:

[0230] According to the configuration of Supplementary Note 17, the same effects as those of the information processing device described in Supplementary Note 1 are achieved.

[0231] (Appendix 18) 18. The information processing method according to claim 17, further comprising: In the threshold value determination step, a second threshold value smaller than the first threshold value is also determined with reference to the comparison result; In the data set generation step, a region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold among one or more inference results by the inference step in the data set after the pseudo label is assigned is determined as a non-learning region that is not to be subjected to learning in the pseudo label reference learning step. 23. An information processing method comprising:

[0232] A region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold tends to have a low-reliability pseudo label even if the region is assigned a pseudo label. By setting such a region as a non-learning region, re-learning can be performed using a pseudo label with a relatively high reliability, so that the configuration of appendix 18 can improve the detection accuracy of the target image detection model in the pseudo label reference learning process.

[0233] (Appendix 19) acquiring a target image; Detecting an object included in the target image using a detection model for the target image; Including, The detection model for the target image is A learning process for learning a detection model using a first data set; a threshold determination process for determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct answer labels attached to each of the one or more images; an inference process for obtaining one or more inference results for each of one or more images included in a second data set by inputting each of the one or more images into the detection model; a data set generation process for generating a data set after a pseudo label is assigned by setting an inference result having a reliability equal to or higher than the threshold value among one or more inference results by the inference process as a pseudo label and associating the pseudo label with a corresponding image; and A pseudo-label reference learning process for learning the detection model for the target image by referring to the dataset after the pseudo labels are added. It was learned by 23. An information processing method comprising:

[0234] According to the configuration of Supplementary Note 19, the same advantageous effects as those of the information processing device described in Supplementary Note 15 are achieved.

[0235] (Appendix 20) A learning process for learning a detection model using a first dataset; a threshold determination step of determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; an inference step of inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating step of generating a pseudo-labeled dataset by setting an inference result having a reliability equal to or higher than the threshold value among one or more inference results obtained by the inference step as a pseudo label and associating the pseudo label with a corresponding image; A pseudo-label reference learning process for learning a target image detection model for detecting an object included in the target image using the pseudo-labeled dataset; A method for manufacturing a detection model, comprising:

[0236] According to the configuration of Supplementary Note 20, a threshold for setting a pseudo label is automatically determined based on a comparison between an inference result of an image included in an evaluation dataset by a detection model trained using a first dataset and a ground truth label associated with the image. Therefore, according to the configuration of Supplementary Note 20, it is possible to reduce the cost related to adjusting the threshold. And, according to the configuration of Supplementary Note 20, a target image detection model is trained using a dataset after pseudo labels are added. Therefore, according to the configuration of Supplementary Note 20, it is possible to manufacture a target image detection model by reducing the cost related to adjusting the threshold. As a result, it is possible to reduce the cost until the target image detection model is trained. Also, if an appropriate value can be determined as the threshold, it is possible to reduce the number of times the threshold is adjusted, and the number of times of training required each time the threshold is adjusted can be reduced. As a result, it is possible to reduce the time until the target image detection model is trained.

[0237] (Appendix 21) A method for producing a detection model according to claim 20, comprising: In the threshold value determination step, a second threshold value smaller than the first threshold value is also determined with reference to the comparison result; In the data set generation step, a region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold among one or more inference results by the inference step in the data set after the pseudo label is assigned is determined as a non-learning region that is not to be subjected to learning in the pseudo label reference learning step. A method for manufacturing a detection model.

[0238] A region corresponding to an inference result having a reliability lower than the first threshold and equal to or higher than the second threshold tends to have a low-reliability pseudo label even if the region is assigned a pseudo label. By setting such a region as a non-learning region, re-learning can be performed using a pseudo label with a relatively high reliability, so that the configuration of appendix 21 can improve the detection accuracy of the target image detection model in the pseudo label reference learning process.

[0239] (Appendix 22) A program for causing a computer to function as an information processing device, the program comprising: A learning means for learning a detection model using a first data set; a threshold determination means for determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; inference means for inputting each of one or more images included in a second data set into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating means for generating a dataset after the pseudo labels are assigned by setting an inference result having a reliability equal to or higher than the threshold value among one or more inference results by the inference means as a pseudo label and associating the pseudo label with a corresponding image; A program that functions as a

[0240] According to the configuration of Supplementary Note 22, the same effects as those of the information processing device described in Supplementary Note 1 are achieved.

[0241] (Appendix 23) A program for causing a computer to function as an information processing device, the program comprising: An acquisition means for acquiring a target image; A detection means for detecting an object included in a target image by using a detection model for the target image; Function as a The detection model for the target image is A learning process for learning a detection model using a first data set; a threshold determination process for determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct answer labels attached to each of the one or more images; an inference process for obtaining one or more inference results for each of one or more images included in a second data set by inputting each of the one or more images into the detection model; a data set generation process for generating a data set after a pseudo label is assigned by setting an inference result having a reliability equal to or higher than the threshold value among one or more inference results by the inference process as a pseudo label and associating the pseudo label with a corresponding image; and A pseudo-label reference learning process for learning the detection model for the target image by referring to the dataset after the pseudo labels are added. A program that is learned by

[0242] According to the configuration of Supplementary Note 23, the same effects as those of the information processing device described in Supplementary Note 15 are achieved.

[0243] [Additional Note 3] A part or all of the above-described embodiments can be further expressed as follows.

[0244] an information processing device comprising at least one processor that executes the following steps: a learning process for learning a detection model using a first dataset; a threshold determination process for determining a first threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more correct labels assigned to each of the one or more images; an inference process for inputting each of one or more images included in a second dataset into the detection model, thereby obtaining one or more inference results for each of the one or more images; and a dataset generation process for generating a dataset after pseudo-labeling by setting, as a pseudo-label, an inference result having a reliability equal to or higher than the first threshold, among the one or more inference results obtained by the inference means, and associating the pseudo-label with a corresponding image.

[0245] The information processing device may further include a memory that stores a program for causing the processor to execute the learning process, the threshold determination process, the inference process, and the data set generation process. The program may be recorded in a computer-readable, non-transitory, tangible recording medium.

[0246] an information processing apparatus comprising at least one processor, the processor executing an acquisition process for acquiring a target image, and a detection process for detecting an object included in the target image using a detection model for the target image, the detection model for the target image being trained through a learning process for training a detection model using a first dataset, a threshold determination process for determining a first threshold by referring to a result of comparing one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model with one or more correct labels assigned to each of the one or more images, an inference process for acquiring one or more inference results for each of the one or more images included in a second dataset into the detection model, a dataset generation process for generating a dataset after pseudo-labeling by setting an inference result having a reliability equal to or higher than the first threshold as a pseudo label and associating the pseudo label with a corresponding image, and a pseudo-label reference learning process for training the detection model for the target image by referring to the dataset after pseudo-labeling.

[0247] The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process and the detection process. The program may be recorded in a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0248] 10, 10a, 10b, 10c, 20, 20a Information processing device 101 Learning Department 101-1 First Study Section 101-2 Second Study Section 102 Threshold determination unit 102-1 First threshold determination unit 102-2 Second threshold determination unit 103 Reasoning part 103-1 First Inference 103-2 Second Inference 104 Dataset Generation Unit 104-1 First data set generation unit 104-2 Second Data Set Generation Unit 105 Re-learning section 106 Learning non-implementation area determination unit 106-1 First non-learning area determination unit 106-2 Second non-learning area determination section 201 Acquisition Department 202 Detection unit DS1 Dataset 1 DS1' Data Set 1' DS2 Dataset 2 DS2' Data Set 2' DSE evaluation dataset DSE1 Evaluation Dataset 1 DSE2 Evaluation Dataset 2 DM Object Detection Model

Claims

1. A learning means for learning a detection model using a first data set; a threshold determination means for determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in an evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; inference means for inputting each of one or more images included in a second data set into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating means for generating a dataset after the pseudo labels are assigned by setting an inference result having a reliability equal to or higher than the first threshold value among one or more inference results by the inference means as a pseudo label and associating the pseudo label with a corresponding image; An information processing device comprising:

2. The system further includes a pseudo-label reference learning means for learning a target image detection model for detecting an object included in the target image by using the pseudo-labeled dataset.

2. The information processing apparatus according to claim 1,

3. The pseudo-label reference learning means re-learns the detection model as the learning of the detection model for the target image.

3. The information processing apparatus according to claim 2.

4. a first learning means for learning a first detection model using a first data set; a second learning means for learning a second detection model using a second data set; a first threshold determination means for determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a first evaluation dataset into the first detection model and one or more correct labels attached to each of the one or more images; a second threshold determination means for determining a second threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a second evaluation data set into the second detection model and one or more correct labels attached to each of the one or more images; a first inference means for inputting each of one or more images included in the second dataset into the first detection model to obtain one or more inference results for each of the one or more images; a second inference means for inputting each of one or more images included in the first dataset into the second detection model to obtain one or more inference results for each of the one or more images; a first dataset generation means for generating a second dataset after the pseudo-label is assigned by setting an inference result having a reliability equal to or higher than the first threshold value as a pseudo label among one or more inference results by the first inference means and associating the pseudo label with a corresponding image; a second dataset generation means for generating a first dataset after the pseudo-label is assigned by setting an inference result having a reliability equal to or higher than the second threshold value as a pseudo label among one or more inference results by the second inference means and associating the pseudo label with a corresponding image; An information processing device comprising:

5. An acquisition means for acquiring a target image; a detection means for detecting an object included in the target image by using a detection model for the target image, the detection model receiving the target image and outputting an inference result regarding the object included in the target image; Further comprising: The target image detection model is trained by a pseudo-label reference training process that trains the target image detection model by referring to the pseudo-labeled dataset.

4. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. At least one processor, A learning step of learning a detection model using a first data set; a threshold determination step of determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; an inference step of inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating step of setting an inference result having a reliability equal to or higher than the first threshold value as a pseudo label among one or more inference results obtained by the inference step, and associating the pseudo label with a corresponding image, thereby generating a dataset after the pseudo label has been assigned; An information processing method comprising:

7. The at least one processor, acquiring a target image; Detecting an object included in the target image using a target image detection model that receives the target image as an input and outputs an inference result regarding the object included in the target image; Further comprising: The target image detection model is trained by a pseudo-label reference training process that trains the target image detection model by referring to the pseudo-labeled dataset.

7. The information processing method according to claim 6,

8. At least one processor, A learning step of learning a detection model using a first data set; a threshold determination step of determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; an inference step of inputting each of one or more images included in a second dataset into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating step of generating a pseudo-labeled dataset by setting an inference result having a reliability equal to or higher than the first threshold value among one or more inference results obtained by the inference step as a pseudo label and associating the pseudo label with a corresponding image; A pseudo-label reference learning process for learning a target image detection model for detecting an object included in the target image using the pseudo-labeled dataset; A method for manufacturing a detection model, comprising:

9. A program for causing a computer to function as an information processing device, the program comprising: A learning means for learning a detection model using a first data set; a threshold determination means for determining a threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in the evaluation dataset into the detection model and one or more correct labels attached to each of the one or more images; inference means for inputting each of one or more images included in a second data set into the detection model to obtain one or more inference results for each of the one or more images; a dataset generating means for generating a dataset after the pseudo labels are assigned by setting an inference result having a reliability equal to or higher than the threshold value among one or more inference results by the inference means as a pseudo label and associating the pseudo label with a corresponding image; A program that functions as a

10. The computer, An acquisition means for acquiring a target image; a detection means for detecting an object included in the target image by using a detection model for the target image, the detection model receiving the target image and outputting an inference result regarding the object included in the target image; Further functioning as The target image detection model is trained by a pseudo-label reference training process that trains the target image detection model by referring to the pseudo-labeled dataset. The program according to claim 9.

11. At least one processor, A first learning step of learning a first detection model using a first dataset; a second learning step of learning a second detection model using a second dataset; a first threshold determination step of determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a first evaluation dataset into the first detection model and one or more correct labels attached to each of the one or more images; a second threshold determination step of determining a second threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a second evaluation dataset into the second detection model and one or more correct labels attached to each of the one or more images; a first inference step of inputting each of one or more images included in the second dataset into the first detection model to obtain one or more inference results for each of the one or more images; a second inference step of inputting each of one or more images included in the first dataset into the second detection model to obtain one or more inference results for each of the one or more images; a first dataset generating step of setting an inference result having a reliability equal to or higher than the first threshold value as a pseudo label among one or more inference results obtained by the first inference step, and associating the pseudo label with a corresponding image, thereby generating a second dataset after the pseudo label is assigned; a second data set generating step of setting an inference result having a reliability equal to or higher than the second threshold value as a pseudo label among one or more inference results obtained by the second inference step, and associating the pseudo label with a corresponding image, thereby generating a first data set after the pseudo label is assigned; An information processing method comprising:

12. A program for causing a computer to function as an information processing device, comprising: a first learning means for learning a first detection model using a first data set; a second learning means for learning a second detection model using a second data set; a first threshold determination means for determining a first threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a first evaluation dataset into the first detection model and one or more correct labels attached to each of the one or more images; a second threshold determination means for determining a second threshold by referring to a comparison result between one or more inference results obtained by inputting each of one or more images included in a second evaluation data set into the second detection model and one or more correct labels attached to each of the one or more images; a first inference means for inputting each of one or more images included in the second dataset into the first detection model to obtain one or more inference results for each of the one or more images; a second inference means for inputting each of one or more images included in the first dataset into the second detection model to obtain one or more inference results for each of the one or more images; a first dataset generation means for generating a second dataset after the pseudo-label is assigned by setting an inference result having a reliability equal to or higher than the first threshold value as a pseudo label among one or more inference results by the first inference means and associating the pseudo label with a corresponding image; a second dataset generation means for generating a first dataset after the pseudo-label is assigned by setting an inference result having a reliability equal to or higher than the second threshold value as a pseudo label among one or more inference results by the second inference means and associating the pseudo label with a corresponding image; A program that functions as a

Citation Information

Patent Citations

  • Situation recognition device, situation recognition method, and situation recognition program

    JP2020197974A

  • Method for determining priority in labeling data

    JP2021022380A

  • Model training using a teacher-student learning paradigm

    US20200410388A1

  • Analysis device, analysis method, and analysis program

    WO2020225923A1