Training evaluation assistance device, composite product generation device, training evaluation assistance method, method for generating composite product, and program

Abstract

Provided is a training evaluation assistance device comprising: a composite product acquisition unit that acquires a plurality of composite products composited using compositing-use products selected from a plurality of original products; a correspondence information acquisition unit that acquires product correspondence information in which the compositing history of each among the plurality of composite products and a composite product corresponding to said compositing history are associated with each other; and a sorting unit that, on the basis of the product correspondence information, sorts a plurality of to-be-sorted products including the plurality of composite products into training-use products and evaluation-use products.

Description

Learning assessment support device, composite product generation device, learning assessment support method, composite product generation method and program

[0001] The present invention relates to a learning assessment support device, a composite product generation device, a learning assessment support method, a composite product generation method, and a program.

[0002] Patent Document 1 describes "a learning evaluation means for performing learning of an image recognition model and evaluation of the image recognition model after learning, using an image dataset including images synthesized by the synthesis means." [Prior art documents] [Patent documents] Patent Document 1: JP 2023-146528 A

[0003] It is preferable to divide the data to be sorted into learning data and evaluation data. General disclosure

[0004] In a first aspect of the present invention, a learning assessment support device is provided, which includes a composite product acquisition unit that acquires multiple composite products synthesized using synthesis products selected from multiple original products, a correspondence information acquisition unit that acquires product correspondence information that associates the synthesis history of each of the multiple composite products with the composite product corresponding to the synthesis history, and an allocation unit that allocates multiple allocation target products including the multiple composite products into learning products and assessment products based on the product correspondence information.

[0005] In the learning assessment support device, the synthesis history may be incorporated into the corresponding synthesis product.

[0006] Any of the above learning assessment support devices may include a storage unit that stores the synthesis history and the synthesis product corresponding to the synthesis history in association with each other.

[0007] In any of the above learning assessment support devices, the allocation unit may allocate the synthesis product to either the learning product or the evaluation product without separating the synthesis product from a synthesized product synthesized using the synthesis product.

[0008] Any of the above learning assessment support devices may include a distribution acquisition unit that acquires a distribution of the plurality of allocation target products with respect to a predetermined index, and the allocation unit may allocate the plurality of composite products to the learning products and the evaluation products based on the distribution of the plurality of allocation target products.

[0009] Any of the above learning assessment support devices may include a similarity evaluation unit that evaluates the similarity between the plurality of allocation target products, and the allocation unit may sort the plurality of allocation target products into the study products and the evaluation products based on the similarity.

[0010] In any of the above learning assessment support devices, the allocating unit may control allocation of the plurality of composite products to a plurality of used products used to synthesize any of the plurality of composite products. The allocating unit may randomly allocate a plurality of unused products among the plurality of original products that are not used to synthesize any of the plurality of composite products according to the number of products.

[0011] The learning assessment support device may further include a proportion acquisition unit that acquires a proportion of the composite portion of each of the plurality of composite products, and the allocating unit may allocate the plurality of allocation target products to the learning products and the evaluation products based on the proportion of the composite portion.

[0012] In any of the above learning assessment support devices, the plurality of composite products may include at least one of images, sounds, music, text, three-dimensional drawing data, videos, movies, and time-series data.

[0013] In any of the above learning assessment support devices, the synthesis history may include information on a synthesis model used to synthesize the plurality of synthesized products.

[0014] In any of the above learning assessment support devices, the product correspondence information may include a relationship between the plurality of composite products and each of the synthesis products used to synthesize the plurality of composite products.

[0015] Any of the above learning assessment support devices may include a tolerance determination unit that determines a tolerance level for allocation by the allocation unit, and the allocation unit may allocate the plurality of allocation target products into the study products and the evaluation products based on the tolerance.

[0016] In any of the above learning evaluation support devices, the plurality of composite products may include a defective product, and when the plurality of original products includes a non-defective product, the tolerance determination unit may allow the non-defective product to be assigned to both the learning product and the evaluation product.

[0017] Any of the above learning assessment support devices may include a learning control unit that trains a predetermined machine learning model using the learning product. Any of the above learning assessment support devices may include a model evaluation unit that evaluates the machine learning model trained with the learning product using the evaluation product.

[0018] In a second aspect of the present invention, there is provided a composite product generation device including an original product acquisition unit that acquires a plurality of original products, and a composite product generation unit that generates a plurality of composite products using a synthesis product selected from the plurality of original products, wherein the composite product generation unit may incorporate a synthesis history into each of the plurality of composite products.

[0019] In a third aspect of the present invention, a learning assessment support method is provided, comprising the steps of: acquiring a plurality of composite products synthesized using a synthesis product selected from a plurality of original products; acquiring product correspondence information that associates the synthesis history of each of the plurality of composite products with the composite product corresponding to the synthesis history; and allocating a plurality of allocation target products including the plurality of composite products into learning products and assessment products based on the product correspondence information.

[0020] In a fourth aspect of the present invention, there is provided a method for generating a composite product, the method comprising: acquiring a plurality of original products; and generating a plurality of composite products using a synthesis product selected from the plurality of original products, the generating a plurality of composite products may include incorporating a synthesis history into each of the plurality of composite products.

[0021] In a fifth aspect of the present invention, a program is provided which, when executed by a computer, causes the computer to acquire a plurality of composite products synthesized using a synthesis product selected from a plurality of original products, acquire product correspondence information that associates the synthesis history of each of the plurality of composite products with the composite product corresponding to the synthesis history, and, based on the product correspondence information, sort a plurality of allocation target products including the plurality of composite products into learning products and evaluation products.

[0022] In a sixth aspect of the present invention, there is provided a program that, when executed by a computer, causes the computer to acquire a plurality of original products and generate a plurality of composite products using a synthesis product selected from the plurality of original products, wherein generating the plurality of composite products may include incorporating a synthesis history into each of the plurality of composite products.

[0023] The above summary of the invention does not list all of the features of the present invention, and subcombinations of these features may also be inventions.

[0024] 5A shows an example of a method for sorting by the learning assessment support device 100 of FIG. 5A. FIG. 6A shows an example of a method for sorting by the learning assessment support device 100 of FIG. 5A. FIG. 7A shows an example of a method for sorting by the learning assessment support device 100 of FIG. 7A. FIG. 8A shows an example of a method for sorting by the learning assessment support device 100 of FIG. 8A. FIG. 9A shows an example of a method for sorting by the learning assessment support device 100 of FIG. 9A. FIG. 10A shows an example of a method for sorting by the learning assessment support device 100 of FIG. 10A. FIG. 11A shows an example of a method for sorting by the learning assessment support device 100 of FIG. 11A. 22 shows an example of a method for allocating allocation target products Pt. FIG. 22 shows an example of a computer 2200 in which aspects of the present invention may be embodied in whole or in part.

[0025] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0026] 1A shows an overview of the configuration of a learning assessment support device 100 and a composite product generation device 200. The learning assessment support device 100 of this example includes a composite product acquisition unit 10, a correspondence information acquisition unit 20, a storage unit 25, and an allocation unit 30. The learning assessment support device 100 may also include a learning control unit 110, a model evaluation unit 120, and a machine learning model 130. The composite product generation device 200 of this example includes an original product acquisition unit 210 and a composite product generation unit 220.

[0027] The composite product generation device 200 generates a composite product PS using the original products Po. The composite product PS may be synthesized using a predetermined synthesis model 225. The synthesis model 225 may include at least one of a generative model, a machine learning model including a multimodal model, or a simulation that generates data. The original products Po used to generate the composite product PS are referred to as synthesis products Ps. The composite product generation device 200 may provide the generated composite product PS to the learning assessment support device 100. The operation of the composite product generation device 200 will be described later.

[0028] The composite product acquisition unit 10 acquires a plurality of composite products PS. The composite product acquisition unit 10 may acquire a plurality of composite products PS from outside the learning assessment support device 100, or may generate and acquire a plurality of composite products PS. The composite product acquisition unit 10 in this example acquires the composite products PS from the composite product generation device 200. The learning assessment support device 100 may be equipped with the composite product generation device 200 and generate the composite products PS.

[0029] The composite product PS may be any product that can be synthesized using any synthesis model 225, and the content of the product is not limited. The composite product PS may include at least one of images, audio, music, text, 3D drawing data, video, movie, and time-series data. For example, the composite product PS may be a defective product image for visual inspection of a product, a medical image for diagnosis, a character image for handwritten character recognition, or a face image for face recognition. The composite product PS may be numerical data for an autonomous driving simulation, synthesized voice data for voice recognition, or synthesized text for training a decision model for emotion analysis. The composite product PS may be synthesized 3D model data for object detection. The composite product PS may be synthesized video for detecting anomalies from surveillance camera footage.

[0030] The correspondence information acquisition unit 20 acquires product correspondence information Ip including a synthesis history Hs of the composite product PS. The product correspondence information Ip may include information about the composite product PS and the products involved in the synthesis of the composite product PS. The correspondence information acquisition unit 20 may store the product correspondence information Ip in the storage unit 25. The correspondence information acquisition unit 20 may provide the product correspondence information Ip to the allocating unit 30 without storing it in the storage unit 25. The correspondence information acquisition unit 20 may acquire the product correspondence information Ip from the composite product generation device 200.

[0031] The synthesis history Hs includes information regarding the synthesis of the composite product PS. The synthesis history Hs may include information regarding the synthesis products Ps used to synthesize the composite product PS. In one example, the synthesis history Hs includes information regarding the original products Po involved in the synthesis of the composite product PS. The synthesis history Hs may include information regarding the synthesis model 225 used to synthesize the composite product PS. The synthesis history Hs may include a learning history of the synthesis model 225 used to synthesize the composite product PS.

[0032] The synthesis history Hs may be incorporated into the corresponding synthesized product PS. The synthesis history Hs may be incorporated into the data of the synthesized product PS itself, or into the metadata of the synthesized product PS. If the synthesized product PS is an image, the synthesis history Hs may be incorporated into the image.

[0033] The product correspondence information Ip may include information associating a synthesis history Hs with a synthesis product PS corresponding to the synthesis history Hs. For example, the product correspondence information Ip may include the relationship between a plurality of synthesis products PS and each synthesis product Ps used to synthesize the plurality of synthesis products PS. In other words, the product correspondence information Ip may include the relationship between a synthesis product PS and an original product Po used to synthesize the synthesis product PS.

[0034] The storage unit 25 may store the product correspondence information Ip. That is, the storage unit 25 may store the synthesis history Hs and the synthesis product PS corresponding to the synthesis history Hs in association with each other. The storage unit 25 may store the product correspondence information Ip acquired by the correspondence information acquisition unit 20. The storage unit 25 may be provided outside the learning assessment support device 100.

[0035] The allocation unit 30 allocates multiple allocation target products Pt into study products PL and evaluation products PE. The allocation target products Pt may include composite products PS. The allocation target products Pt may include both composite products PS and original products Po. The allocation unit 30 of this example allocates multiple allocation target products Pt, including multiple composite products PS, into study products PL and evaluation products PE based on the product correspondence information Ip. The allocation unit 30 may read out and refer to the product correspondence information Ip from the storage unit 25.

[0036] The learning control unit 110 trains a predetermined machine learning model 130 using the training product PL. The learning control unit 110 may train any machine learning model 130 that can learn using the training product PL. The machine learning model 130 may be the same as or different from the composite model 225 used to generate the composite product PS.

[0037] The model evaluation unit 120 evaluates the machine learning model 130 that has been trained with the training product PL, using the evaluation product PE. The model evaluation unit 120 may evaluate the product generated by the machine learning model 130 to evaluate the learning result of the machine learning model 130.

[0038] The original product acquisition unit 210 acquires a plurality of original products Po. The original product acquisition unit 210 provides at least one of the plurality of original products Po to the composite product generation unit 220 as a product to be synthesized Ps.

[0039] The original product Po may be a product that is not generated using the synthetic model 225. The original product Po may be at least one of an image, an audio, a music, a text, three-dimensional drawing data, a video, a movie, and time-series data. The original product Po may be provided by a user. If the original product Po is an image, the original product Po may be a photograph taken by a camera, an image drawn by a person, or an image handwritten by a person.

[0040] The synthesis product Ps is the original product Po used to synthesize the synthesis product PS. All of the multiple original products Po may be used as the synthesis product Ps, or only a portion of the multiple original products Po may be used as the synthesis product Ps.

[0041] The composite product generation unit 220 generates a plurality of composite products PS using a synthesis product Ps selected from a plurality of original products Po. The composite product generation unit 220 may have a synthesis model 225 for generating the composite product PS. The composite product generation unit 220 may have a plurality of synthesis models 225. The synthesis model 225 may or may not be trained using the original products Po.

[0042] The composite product generation unit 220 may create a synthesis history Hs when generating a composite product PS. The composite product generation unit 220 may incorporate the generated synthesis history Hs into the composite product PS. The composite product generation unit 220 may incorporate a synthesis history Hs for the corresponding product to be synthesized Ps into each of the multiple composite products PS. The composite product generation unit 220 may provide the composite product PS incorporating the synthesis history Hs to the learning assessment support device 100.

[0043] The composite product generation unit 220 may provide the synthesis history Hs created when generating the composite product PS to the learning assessment support device 100 without incorporating it into the composite product PS. The composite product generation unit 220 may associate the synthesis history Hs with the composite product PS and provide it to the learning assessment support device 100. The composite product generation unit 220 may provide the learning assessment support device 100 with a list in which the synthesis history Hs and the composite product PS are associated. The learning assessment support device 100 may store the provided synthesis history Hs in the storage unit 25.

[0044] The composite product generation unit 220 provides the generated composite product PS to the learning assessment support device 100. The original product acquisition unit 210 may provide the original product Po to the learning assessment support device 100. The original product acquisition unit 210 may provide the used product Pu used to synthesize the composite product PS to the learning assessment support device 100 as the original product Po, or may provide the unused product Pn not used to synthesize the composite product PS to the learning assessment support device 100. The used product Pu and the unused product Pn will be described later.

[0045] The learning assessment support device 100 of this example can prevent data included in the learning products PL from leaking to the evaluation products PE by allocating multiple allocation target products Pt into learning products PL and evaluation products PE based on the product correspondence information Ip. This allows the learning assessment support device 100 of this example to more accurately evaluate the machine learning model 130 trained using the learning products PL.

[0046] Here, the allocating unit 30 may allocate the combination products Ps to either the learning product PL or the evaluation product PE without separating them from the combined product PS that is combined using the combination products Ps. In other words, the allocating unit 30 may collect multiple allocation target products Pt that include a common combined product PS in the combination history Hs and allocate them to either the learning product PL or the evaluation product PE.

[0047] When the allocation target products Pt are time-series data, the allocating unit 30 may allocate the time-series data so that the time-series data is consistent between the learning products PL and the evaluation products PE. This allows the learning assessment support device 100 to accurately assess the time-series data. The allocating unit 30 may allocate the time-series data so that the time-series data of the multiple allocation target products Pt before allocation is consistent with the time-series data of the learning products PL and the evaluation products PE.

[0048] When the allocation target products Pt include both original products Po and composite products PS, the allocation unit 30 may allocate the products by adjusting the ratio between the original products Po and the composite products PS. For example, the allocation unit 30 may allocate the products Pt so that the ratio between the original products Po and the composite products PS is a predetermined ratio. The allocation unit 30 may allocate the products Pt so that the ratio between the original products Po and the composite products PS is uniform between the learning products PL and the evaluation products PE.

[0049] When the allocation target products Pt are used in cross-validation, the allocation unit 30 may control allocation for each learning evaluation. Cross-validation may refer to performing learning evaluation by swapping at least a portion of the learning products PL and the evaluation products PE. In cross-validation, learning evaluation may be performed by swapping all of the learning products PL and the evaluation products PE. In cross-validation, the allocation target products Pt may be divided into a predetermined number of divisions and allocated to the learning products PL and the evaluation products PE. Even when performing learning evaluation using cross-validation, any of the allocation methods disclosed in this specification may be used.

[0050] 1B is an example of a flowchart for implementing the learning support method. In step S100, a composite product PS is acquired. In step S100, a generated composite product PS may be acquired, or the composite product PS may be generated from an original product Po and acquired.

[0051] In step S102, product correspondence information Ip is acquired. In step S104, a plurality of allocation target products Pt are allocated into study products PL and evaluation products PE. A plurality of allocation target products Pt may be allocated into study products PL and evaluation products PE based on the product correspondence information Ip.

[0052] In step S106, the learning product PL is used to perform learning of the machine learning model 130. In step S108, the evaluation product PE is used to evaluate the learning results of the machine learning model 130. After executing step S108, the process may return to step S104, and the allocation method for the allocation target products Pt may be changed based on the evaluation results of the machine learning model 130. Thereafter, the learning and evaluation of the machine learning model 130 may be repeated. Alternatively, after executing step S108, the process may return to step S100, and a new composite product PS may be generated from the original product Po.

[0053] 1C is a flowchart for realizing a method for generating a composite product. In step S200, a plurality of original products Po are obtained. In step S202, a composite product PS is generated. In step S204, the plurality of original products Po and the composite product PS may be transmitted. After step S204 is performed, step S100 and subsequent steps in FIG. 1B may be performed. The entity that executes the flowchart in FIG. 1B may be the same as or different from the entity that executes the flowchart in FIG. 1C.

[0054] 2A shows an example of a method for allocating allocation target products Pt. In this example, a case where multiple composite products PS(G) to PS(K) are allocated to a training product PL and an evaluation product PE is described. This figure shows the composite product PS with a combination history Hs attached.

[0055] The multiple composite products PS(G) to PS(K) have different composite histories Hs. In this example, the multiple composite products PS(G) to PS(K) differ in the types of composite products Ps used for composition. The composite history Hs illustrated corresponding to each composite product PS indicates the composite product Ps used to compose the composite product PS.

[0056] The composite product PS(G) is synthesized by using the original product Po(A,B) with the composite product Ps. Po(A,B) in the figure indicates that both the original product Po(A) and the original product Po(B) are used in the composite product Ps. The composite product PS(H) is synthesized by using the original product Po(A) with the composite product Ps. The composite product PS(I) is synthesized by using the original product Po(A,B,C) with the composite product Ps. The composite product PS(J) is synthesized by using the original product Po(E,F) with the composite product Ps. The composite product PS(K) is synthesized by using the original product Po(F) with the composite product Ps.

[0057] The learning assessment support device 100 of this example allocates the composite product PS(H) and the composite product PS(I) to the learning product PL. That is, the learning assessment support device 100 allocates the composite product PS, which is created using any of the original products Po(A) to Po(C), to the learning product PL. On the other hand, the learning assessment support device 100 does not allocate the composite product PS, which is created using any of the original products Po(A) to Po(C), to the evaluation product PE.

[0058] Similarly, the learning assessment support device 100 of this example assigns the composite product PS(J) and the composite product PS(K) to the evaluation product PE. That is, the learning assessment support device 100 assigns the composite product PS, which is composed using either the original product Po(E) or the original product Po(F), to the evaluation product PE. On the other hand, the learning assessment support device 100 does not assign the composite product PS, which is composed using either the original product Po(E) or the original product Po(F), to the learning product PL.

[0059] Note that the learning assessment support device 100 of this example may not allocate some of the composite products PS of the allocation target products Pt. The learning assessment support device 100 of this example does not allocate the composite product PS (G) to either the learning product PL or the evaluation product PE. The learning assessment support device 100 of this example does not allocate the original product Po as an allocation target product Pt, but the original product Po may be used as an allocation target product Pt.

[0060] FIG. 2B shows an example of a method for allocating allocation target products Pt. In this example, an example of allocation is described in consideration of information on the composition model 225 used to synthesize the composite product PS. In this example, differences from the allocation method of FIG. 2A are particularly described, and the rest may be the same as in FIG. 2A. The composition history Hs shown corresponding to each composite product PS indicates the composition product Ps and the composition model 225 used to synthesize the composite product PS. In this example, the composite products PS(H), PS(I), and PS(K) include information on the composition model 225 in the composition history Hs.

[0061] In this example, the composite product PS(H) is synthesized by using the original product Po(A) with the product Ps to be synthesized. The composite product PS(H) is also synthesized using a synthesis model M(A,B) that has been trained using the original product Po(A) and the original product Po(B). The synthesis model M(A,B) is an example of the synthesis model 225.

[0062] In this example, the composite product PS(I) is synthesized by using the original product Po(A, B, C) in the synthesis product Ps. The composite product PS(I) is also synthesized using a synthesis model M(E, F) that has been trained using the original product Po(E) and the original product Po(F). The synthesis model M(E, F) is an example of the synthesis model 225.

[0063] In this example, the composite product PS(K) is synthesized by using the original product Po(F) on the synthesis product Ps. The composite product PS(K) is also synthesized using the original product Po(E) and a synthesis model M(E, F) trained using the original product Po(F).

[0064] The learning assessment support device 100 in this example assigns the composite product PS(H) to the study product PL. That is, the learning assessment support device 100 assigns the composite product PS in which the synthesis history Hs is involved with any of the original products Po(A) to Po(C) to the study product PL. On the other hand, the learning assessment support device 100 does not assign the composite product PS in which the synthesis history Hs is involved with any of the original products Po(A) to Po(C) to the evaluation product PE.

[0065] Similarly, the learning assessment support device 100 of this example assigns the composite product PS(J) and the composite product PS(K) to the evaluation product PE. That is, the learning assessment support device 100 assigns the composite product PS in which the synthesis history Hs is involved with either the original product Po(E) or the original product Po(F) to the evaluation product PE. On the other hand, the learning assessment support device 100 does not assign the composite product PS in which the synthesis history Hs is involved with either the original product Po(E) or the original product Po(F) to the learning product PL.

[0066] The learning assessment support device 100 of this example assigns the composite product PS based on which original product Po the composite model 225 used to synthesize the composite product PS was trained with. This prevents products whose synthesis history Hs includes the same original product Po from being assigned to both the learning product PL and the evaluation product PE, resulting in the leakage of learning assessment data, when the machine learning model 130 is trained and evaluated using the composite product PS. Therefore, the learning assessment support device 100 of this example can correctly evaluate the machine learning model 130 trained using the learning product PL.

[0067] FIG. 3A shows an example of a method for allocating allocation target products Pt. In this example, a case will be described in which multiple original products Po(A) to Po(F) and multiple composite products PS(G) to PS(K) are allocated to learning products PL and evaluation products PE. The learning assessment support device 100 of this example differs from the allocation method of FIG. 2B in that multiple original products Po are allocated in addition to multiple composite products PS. In this example, differences from the allocation method of FIG. 2B will be particularly described, and other aspects may be the same as the allocation method of FIG. 2B.

[0068] The learning assessment support device 100 of this example assigns the original products Po(A) to Po(C) and the composite product PS(H) and PS(I) to the learning products PL. That is, the learning assessment support device 100 assigns the original products Po(A) to Po(C) and the composite product PS whose composition history Hs includes any of these to the learning products PL. On the other hand, the learning assessment support device 100 does not assign the original products Po(A) to Po(C) and the composite product PS whose composition history Hs includes any of these to the evaluation products PE.

[0069] Similarly, the learning assessment support device 100 of this example assigns the original products Po(E) and Po(F) and the composite product PS(J) to the evaluation products PE. That is, the learning assessment support device 100 assigns the original products Po(E) and Po(F) and the composite products PS whose synthesis history Hs includes any of these to the evaluation products PE. On the other hand, the learning assessment support device 100 does not assign the original products Po(E) and Po(F) and the composite products PS whose synthesis history Hs includes any of these to the learning products PL.

[0070] In this example, even when multiple original products Po and multiple composite products PS are allocated, the learning evaluation support device 100 can avoid learning data leaking into the evaluation data by taking into account the synthesis history Hs when allocating the products.

[0071] 3B shows an example of a method for allocating allocation target products Pt. The learning assessment support device 100 of this example differs from the allocation method of FIG. 3A in that the composite product PS is not allocated to the evaluation product PE. In this example, differences from the allocation method of FIG. 3A will be particularly described, and other points may be the same as the allocation method of FIG. 3A.

[0072] The learning assessment support device 100 of this example assigns the original product Po(E) and the original product Po(F) to the evaluation product PE. On the other hand, the learning assessment support device 100 does not assign the composite product PS, whose synthesis history Hs includes either the original product Po(E) or the original product Po(F), to the evaluation product PE. In this way, the learning assessment support device 100 may assign the composite product PS only to the learning product PL, and not assign the composite product PS to the evaluation product PE.

[0073] 4A shows a modified example of the learning assessment support device 100. The learning assessment support device 100 of this example differs from the learning assessment support device 100 of FIG. 1A in that it includes a distribution acquisition unit 50. The distribution acquisition unit 50 may also be applied to other examples of the learning assessment support device 100.

[0074] The distribution acquisition unit 50 acquires the distribution of the multiple allocation target products Pt with respect to a predetermined index. The distribution acquisition unit 50 of this example acquires the distribution of the multiple allocation target products Pt before allocating the multiple allocation target products Pt. The distribution acquisition unit 50 may store the distribution of the multiple allocation target products Pt in the storage unit 25.

[0075] The distribution of the allocation target products Pt may be a distribution for any index. The distribution may be a two-dimensional distribution or a three- or more-dimensional distribution. For example, if the allocation target products Pt are products that include defective parts, the distribution of the allocation target products Pt is a distribution of "frequency" for "size of composite defective parts."

[0076] The allocation unit 30 allocates the multiple composite products PS into learning products PL and evaluation products PE based on the distribution of the multiple allocation target products Pt. In one example, the allocation unit 30 allocates the multiple composite products PS into learning products PL and evaluation products PE so that the distribution of the learning products PL approaches the distribution of the multiple allocation target products Pt before allocation, and so that the distribution of the evaluation products PE approaches the distribution of the multiple allocation target products Pt before allocation.

[0077] The allocating unit 30 may allocate the learning products PL so that the distribution of the evaluation products PE approaches the distribution of the learning products PL. That is, the distribution of the learning products PL and the distribution of the evaluation products PE may be the same as or different from the distribution of the multiple allocation target products Pt before allocation. In this case, the distribution acquisition unit 50 may or may not acquire the distribution of the multiple allocation target products Pt before allocation.

[0078] 4B shows an example of a sorting method by the learning assessment support device 100 of FIG. 4A. The learning assessment support device 100 of this example sorts the sorting target products Pt into learning products PL and evaluation products PE based on the distribution of "frequency" with respect to "size of poorly combined parts." The learning assessment support device 100 of this example sorts the sorting target products Pt so that the distribution of the learning products PL and the distribution of the evaluation products PE become closer.

[0079] 5A shows a modified example of the learning assessment support device 100. The learning assessment support device 100 of this example differs from the learning assessment support device 100 of FIG. 1A in that it includes a similarity evaluation unit 60. The similarity evaluation unit 60 may also be applied to other examples of the learning assessment support device 100.

[0080] The similarity evaluation unit 60 evaluates the similarity between multiple allocation target products Pt. In one example, the similarity evaluation unit 60 calculates and evaluates the similarity of the content of multiple allocation target products Pt. The similarity of the allocation target products Pt may be the similarity of the products themselves, and does not have to be the similarity of the synthesis history Hs. For example, the similarity of the allocation target products Pt is the similarity of defective parts when the products are images of defective products. The similarity of the allocation target products Pt may be the similarity of the product content, or may be cosine similarity, feature point similarity, histogram similarity, or product style similarity.

[0081] The allocating unit 30 allocates the multiple allocation target products Pt into study products PL and evaluation products PE based on the similarity evaluated by the similarity evaluation unit 60. When the similarity between the multiple allocation target products Pt is high, the allocating unit 30 may allocate the products equally into study products PL and evaluation products PE. When the similarity between the multiple allocation target products Pt is low, the allocating unit 30 may allocate the products to either study products PL or evaluation products PE without separating them. The magnitude of the similarity may be determined based on whether it is larger or smaller than a predetermined threshold. The size of the threshold may be determined according to the number of allocation target products Pt or the required accuracy of learning evaluation.

[0082] On the other hand, if the similarity of the synthesis histories Hs is high, the allocating unit 30 may allocate the product to either the learning product PL or the evaluation product PE without separating the products. Also, if the similarity of the synthesis histories Hs is low, the allocating unit 30 may allocate the product to either the learning product PL or the evaluation product PE separately.

[0083] 5B shows an example of an allocation method by the learning assessment support device 100 of FIG. 5A. In this example, when the similarity between multiple allocation target products Pt is high, the learning assessment support device 100 allocates the products equally to study products PL and evaluation products PE. For example, because the allocation target product Pt1 has a high similarity to the allocation target product Pt2, the learning assessment support device 100 allocates the allocation target product Pt1 to the study products PL and allocates the allocation target product Pt2 to the evaluation products PE.

[0084] 5C shows an example of an allocation method by the learning assessment support device 100 of FIG. 5A. In the present example, when the similarity between multiple allocation target products Pt is low, the learning assessment support device 100 does not separate the products but allocates them to either the study products PL or the evaluation products PE. For example, because the similarity between the allocation target product Pt3 and the allocation target product Pt4 is low, the learning assessment support device 100 allocates both the allocation target product Pt3 and the allocation target product Pt4 to the study products PL.

[0085] The learning assessment support device 100 may also sort the allocation target products Pt according to the degree of similarity when the allocation target products Pt include allocation target products Pt1 and Pt2 with high similarity and allocation target products Pt3 and Pt4 with low similarity. In other words, the learning assessment support device 100 may divide the multiple allocation target products Pt according to the degree of similarity and sort each product according to different criteria.

[0086] 6A shows a modified example of the learning assessment support device 100. The learning assessment support device 100 of this example differs from the learning assessment support device 100 of FIG. 1A in that it includes a ratio acquisition unit 70. The ratio acquisition unit 70 may also be applied to other examples of the learning assessment support device 100.

[0087] The proportion acquisition unit 70 acquires the proportion of the combined portion Rs in the product for each of the multiple combined products PS. The proportion acquisition unit 70 may automatically acquire the proportion of the combined portion Rs in the product from the combined product PS, or may acquire the proportion of the combined portion Rs from information assigned to the combined product PS at the time of generation. The proportion acquisition unit 70 may also acquire the proportion of the combined portion Rs from information in which the user manually specifies the combined portion Rs.

[0088] The proportion of the composite portion Rs in the product may be the proportion of the composite portion Rs to the entire product when the composite product PS contains the composite portion Rs and a non-composite portion where the original product Po remains. For example, when a defective product image is generated by adding a defective portion to a non-defective product image, the defective portion is the composite portion Rs, and the rest of the image is the non-composite portion. In the case of a moving image, the proportion of the composite portion Rs in the product may be the length of the moving image of the composite portion Rs relative to the length of the entire moving image.

[0089] The allocating unit 30 allocates the multiple allocation target products Pt into study products PL and evaluation products PE based on the ratio of the combined portion Rs. The allocating unit 30 may acquire from the ratio acquiring unit 70 the ratio of the combined portion Rs in each of the multiple combined products PS.

[0090] 6B shows an example of an allocation method by the learning assessment support device 100 of FIG. 6A. The learning assessment support device 100 of this example allocates multiple allocation target products Pt into study products PL and evaluation products PE based on the proportion of the combined portion Rs of the multiple allocation target products Pt. In one example, the learning assessment support device 100 allocates a combined product PS1 having a larger proportion of the combined portion Rs than a predetermined standard to the study product PL, and allocates a combined product PS2 having a smaller proportion of the combined portion Rs than the predetermined standard to the evaluation product PE. For example, the learning assessment support device 100 determines that a proportion of the combined portion Rs is large when it exceeds 10%, and determines that a proportion of the combined portion Rs is small when it is 10% or less.

[0091] Although the learning assessment support device 100 in this example allocates the composite product PS with a small proportion of the composite portion Rs to the evaluation product PE, it may also allocate the composite product PS with a large proportion of the composite portion Rs to the evaluation product PE. Note that the learning assessment support device 100 may allocate the composite product PS so that the proportions of the composite portion Rs are uniform between the learning product PL and the evaluation product PE.

[0092] 7A shows a modified example of the learning assessment support device 100. The learning assessment support device 100 of this example differs from the learning assessment support device 100 of Fig. 1A in that it includes a tolerance determination unit 80. The tolerance determination unit 80 may also be applied to other examples of the learning assessment support device 100.

[0093] The tolerance determination unit 80 determines the tolerance of the allocation level by the allocation unit 30. The tolerance determination unit 80 may allow an allocation target product Pt that satisfies predetermined conditions to be allocated to both a study product PL and an evaluation product PE. The tolerance determination unit 80 may determine the tolerance of the allocation level in accordance with instructions input by the user.

[0094] The allocation unit 30 allocates a plurality of allocation target products Pt into study products PL and evaluation products PE based on the tolerance determined by the tolerance determination unit 80. In one example, the allocation unit 30 allocates allocation target products Pt whose tolerance meets a predetermined standard to both study products PL and evaluation products PE, and allocates allocation target products Pt whose tolerance does not meet the predetermined standard to either study products PL or evaluation products PE.

[0095] The learning assessment support device 100 in this example allocates allocation target products Pt whose tolerance meets a predetermined standard to both learning products PL and evaluation products PE, thereby increasing the number of learning products PL and evaluation products PE even when the number of allocation target products Pt is small, thereby improving the efficiency of learning assessment.

[0096] 7B shows an example of a sorting method used by the learning assessment support device 100 of FIG. 7A. In this example, a case will be described in which the sorting target products Pt include good products Pg and defective products Pb. The learning assessment support device 100 of this example changes the sorting method for good products Pg and defective products Pb.

[0097] When a plurality of original products Po include a good product Pg, the tolerance determination unit 80 allows the good product Pg to be allocated to both the learning product PL and the evaluation product PE. On the other hand, when a plurality of combined products PS include a defective product Pb, the tolerance determination unit 80 allocates the defective product Pb to either the learning product PL or the evaluation product PE, but does not allow it to be allocated to both.

[0098] In this example, the multiple allocation target products Pt include good products Pg(A) to Pg(H) and defective products Pb(A) to Pb(H). The learning assessment support device 100 may allocate the good products Pg(A) to Pg(H) to both the learning products PL and the evaluation products PE. On the other hand, the learning assessment support device 100 may allocate the defective products Pb(A) to Pb(D) to the learning products PL and the defective products Pb(E) to Pb(H) to the evaluation products PE. The learning assessment support device 100 may allocate the defective products Pb(A) to Pb(H) so that the number of products is equal.

[0099] 8 shows an example of a method for allocating allocation target products Pt. The learning assessment support device 100 of this example changes the allocation method for used products Pu and unused products Pn. The allocation method of this example may be executed in combination with other allocation methods.

[0100] The used product Pu and the unused product Pn are examples of the original product Po. The used product Pu is a product used to synthesize one of the multiple composite products PS. In other words, the used product Pu is used as a synthesis product Ps when synthesizing one of the composite products PS. The used product Pu only needs to be used as a synthesis product Ps for one of the composite products PS, and does not necessarily have to be used as a synthesis product Ps when synthesizing another composite product PS.

[0101] An unused product Pn is an original product Po that has not been used in the synthesis of any of the composite products PS. In other words, since the unused product Pn has not been used in the synthesis of any of the composite products PS, it is not included in the synthesis history Hs of the composite products PS. Therefore, it may be determined that the unused product Pn does not cause a leak, regardless of the allocation destination.

[0102] The allocation unit 30 may control allocation of the multiple use products Pu using a predetermined allocation method. The allocation unit 30 may control allocation of the multiple composite products PS and the multiple use products Pu. Controlling allocation may refer to allocating according to predetermined rules so as to prevent leakage of learning assessment data. Controlling allocation may refer to allocating using the allocation method disclosed in this specification so as to prevent leakage of learning assessment data.

[0103] For example, the allocating unit 30 allocates the use products Pu(A) to Pu(F) of the use products Pu(A) to Pu(H) to the learning product PL and allocates the use products Pu(G) and Pu(H) to the evaluation product PE. Furthermore, the allocating unit 30 in this example allocates the composite products PS(A) to PS(F) of the composite products PS(A) to PS(H) to the learning product PL and allocates the composite products PS(G) and PS(H) to the evaluation product PE.

[0104] The allocating unit 30 may randomly allocate the unused products Pn according to the number of products. In one example, the allocating unit 30 allocates the unused products Pn to the learning products PL and the evaluation products PE so that the number of products is equal. In other words, the allocation of the unused products Pn that do not cause leaks does not need to be controlled.

[0105] In this example, the allocation unit 30 allocates unused products Pn(A) to Pn(D) out of the unused products Pn(A) to Pn(H) to the learning product PL, and allocates unused products Pn(E) to Pn(H) to the evaluation product PE.

[0106] As described above, the learning assessment support device 100 can improve the accuracy of learning assessment by sorting multiple allocation target products Pt based on the product correspondence information Ip. The learning assessment support device 100 may execute an appropriate combination of the sorting methods disclosed in this specification. The learning assessment support device 100 may divide multiple allocation target products Pt into multiple groups and sort the products Pt using a different sorting method for each divided group.

[0107] 9 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0108] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0109] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0110] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0111] ROM 2230 stores therein a boot program or the like that is executed by computer 2200 upon activation, and / or programs that depend on the hardware of computer 2200. I / O chip 2240 may also connect various I / O units to I / O controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0112] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0113] For example, when communication is performed between computer 2200 and an external device, CPU 2212 may execute a communication program loaded into RAM 2214 and instruct communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in RAM 2214, hard disk drive 2224, DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0114] Furthermore, the CPU 2212 may cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and may perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0115] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0116] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0117] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0118] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0119] 10... Composite product acquisition unit, 20... Correspondence information acquisition unit, 25... Memory unit, 30... Allocation unit, 50... Distribution acquisition unit, 60... Similarity evaluation unit, 70... Ratio acquisition unit, 80... Tolerance determination unit, 100... Learning evaluation support device, 110... Learning control unit, 120... Model evaluation unit, 130... Machine learning model, 200... Composite product generation device, 210... Original product acquisition unit, 220... Composite product generation unit, 225... Composite model, 2 200...Computer, 2201...DVD-ROM, 2210...Host controller, 2212...CPU, 2214...RAM, 2216...Graphics controller, 2218...Display device, 2220...Input / output controller, 2222...Communication interface, 2224...Hard disk drive, 2226...DVD-ROM drive, 2230...ROM, 2240...Input / output chip, 2242...Keyboard

Claims

1. A learning and evaluation support device comprising: a synthetic product acquisition unit that acquires a plurality of synthetic products synthesized using synthetic products selected from a plurality of original products; a correspondence information acquisition unit that acquires product correspondence information associating the synthesis history of each of the plurality of synthetic products with the synthetic product corresponding to the synthesis history; and a distribution unit that distributes a plurality of products to be distributed including the plurality of synthetic products into learning products and evaluation products based on the product correspondence information.

2. The learning and evaluation support device according to claim 1, wherein the synthesis history is incorporated into the corresponding synthetic product.

3. The learning and evaluation support device according to claim 1, further comprising a storage unit that stores the synthesis history and the synthetic product corresponding to the synthesis history in association with each other.

4. The learning and evaluation support device according to claim 1, wherein the distribution unit distributes the synthetic product and the synthetic product synthesized using the synthetic product without distinction to either the learning product or the evaluation product.

5. The learning and evaluation support device according to claim 1, further comprising a distribution acquisition unit that acquires the distribution of the plurality of products to be distributed with respect to a predetermined index, wherein the distribution unit distributes the plurality of synthetic products into the learning products and the evaluation products based on the distribution of the plurality of products to be distributed.

6. The learning and evaluation support device according to any one of claims 1 to 5, further comprising a similarity evaluation unit that evaluates the similarity between the plurality of products to be distributed, wherein the distribution unit distributes the plurality of products to be distributed into the learning products and the evaluation products based on the similarity.

7. The learning and evaluation support device according to any one of claims 1 to 5, wherein the distribution unit controls the distribution of the plurality of synthetic products and the plurality of used products used in the synthesis of any one of the plurality of synthetic products, and randomly distributes a plurality of unused products that are not used in the synthesis of any of the plurality of synthetic products among the plurality of original products according to the number of products.

8. A ratio acquisition unit that acquires the ratio of the synthesized part in the product for each of the plurality of synthesized products, and the distribution unit distributes the plurality of products to be distributed to the learning product and the evaluation product based on the ratio of the synthesized part. The learning and evaluation support device according to any one of claims 1 to 5.

9. The plurality of synthesized products include at least one of an image, voice, music, text, 3D drawing data, video, movie, and time series data. The learning and evaluation support device according to any one of claims 1 to 5.

10. The synthesis history includes information on the synthesis model used for the synthesis of the plurality of synthesized products. The learning and evaluation support device according to any one of claims 1 to 5.

11. The product correspondence information includes the relationship between the plurality of synthesized products and each of the synthesis products used for the synthesis of the plurality of synthesized products. The learning and evaluation support device according to any one of claims 1 to 5.

12. A tolerance determination unit that determines the tolerance of the distribution level by the distribution unit, and the distribution unit distributes the plurality of products to be distributed to the learning product and the evaluation product based on the tolerance. The learning and evaluation support device according to any one of claims 1 to 5.

13. The plurality of synthesized products include defective products, and when the plurality of original products include non-defective products, the tolerance determination unit allows the non-defective products to be distributed to both the learning product and the evaluation product. The learning and evaluation support device according to claim 12.

14. A learning control unit that causes a predetermined machine learning model to be learned using the learning product, and a model evaluation unit that evaluates the machine learning model learned with the learning product using the evaluation product. The learning and evaluation support device according to any one of claims 1 to 5.

15. An original product acquisition unit that acquires a plurality of original products, and a synthesized product generation unit that generates a plurality of synthesized products using the synthesis products selected from the plurality of original products, and the synthesized product generation unit incorporates a synthesis history into each of the plurality of synthesized products. A synthesized product generation device.

16. A learning and evaluation support method comprising: obtaining a plurality of synthesized products synthesized using synthesized products selected from a plurality of original products; obtaining product correspondence information associating the synthesis history of each of the plurality of synthesized products with the synthesized product corresponding to the synthesis history; and allocating a plurality of products to be allocated including the plurality of synthesized products into learning products and evaluation products based on the product correspondence information.

17. A method for generating synthesized products, comprising: a computer obtaining a plurality of original products; and the computer generating a plurality of synthesized products using synthesized products selected from the plurality of original products, wherein the step of generating the plurality of synthesized products includes the step of incorporating a synthesis history into each of the plurality of synthesized products.

18. A program which, when executed by a computer, causes the computer to obtain a plurality of synthesized products synthesized using synthesized products selected from a plurality of original products, obtain product correspondence information associating the synthesis history of each of the plurality of synthesized products with the synthesized product corresponding to the synthesis history, and allocate a plurality of products to be allocated including the plurality of synthesized products into learning products and evaluation products based on the product correspondence information.

19. A program which, when executed by a computer, causes the computer to obtain a plurality of original products, generate a plurality of synthesized products using synthesized products selected from the plurality of original products, and generating the plurality of synthesized products includes incorporating a synthesis history into each of the plurality of synthesized products.

Images