Data set creation device, data set creation method, and program

Abstract

In this dataset creation device, an acquisition means acquires imaged images of an object. A data processing means generates sample images from the imaged images, said sample images being images of sections of the object, and generates a training dataset including a sample image among the sample images that is useful for satisfying a prescribed criterion. A data quality estimation means estimates the quality of the dataset on the basis of the imaged images and the dataset. A display control means displays, on a display device, imaging assistance information including dataset quality information.

Description

[Technical field]

[0001] The present disclosure relates to creating training datasets. [Background technology]

[0002] There is a known method for identifying products in stores by photographing them and performing image recognition. To perform image recognition for products, it is necessary to train a recognition model for the target product. In addition, when a new product is released, it is necessary to train an existing recognition model so that the new product can be recognized. To train a recognition model, it is necessary to prepare a learning dataset for the target product.

[0003] Patent Document 1 describes a method for evaluating labelers in order to improve the quality of data sets created for machine learning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 187421 Summary of the Invention [Problem to be solved by the invention]

[0005] When creating a learning dataset, the appearance of a target product or other object is photographed from various angles, and sample images for learning are generated from the photographed images. However, experience is required to take images that can efficiently create data that is useful for learning. Specifically, those with little experience have issues such as not knowing how much data is needed to train a recognition model, and not being able to collect variations in appearance that are useful for learning.

[0006] One objective of the present disclosure is to provide a dataset creation device that enables even a person with no particular experience to capture images suitable for generating training data and create a high-quality training dataset. [Means for solving the problem]

[0007] In one aspect of the present disclosure, a data set creation device includes: An acquisition means for acquiring a photographed image of an object; a data processing means for generating sample images, which are images of portions of the object, from the captured images, and generating a learning dataset including valid sample images that satisfy a predetermined criterion among the sample images; a data quality estimation means for estimating a quality of the data set based on the captured image and the data set; Quality information for the dataset the captured image, information indicating a position from which the sample image is cut out in the captured image, valid sample images included in the data set, and a valid image rate which is a ratio of the valid sample images to the total number of the sample images; A display control means for displaying, on a display device, photographing support information including the Equipped with. In addition, according to one aspect of the present disclosure, a data set creation device includes: An acquisition means for acquiring a photographed image of an object; a data processing means for generating sample images, which are images of portions of the object, from the captured images, and generating a learning dataset including valid sample images that satisfy a predetermined criterion among the sample images; a data quality estimation means for estimating a quality of the data set based on the captured image and the data set; a display control means for displaying on a display device photographing support information including quality information of the data set and an achievement level indicating a ratio of the number of valid sample images already obtained to the total number of images required for learning; Equipped with.

[0008] In another aspect of the present disclosure, a method for creating a data set includes: 1. A computer implemented method for creating a data set, comprising: Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; Quality information for the dataset the captured image, information indicating a position from which the sample image is cut out in the captured image, valid sample images included in the data set, and a valid image rate which is a ratio of the valid sample images to the total number of the sample images; The photographing support information including the above is displayed on a display device. In another aspect of the present disclosure, a data set creation method includes: 1. A computer implemented method for creating a data set, comprising: Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; Shooting support information including quality information of the data set and an achievement level indicating the ratio of the number of valid sample images already obtained to the total number of images required for learning is displayed on a display device.

[0009] In yet another aspect of the disclosure, a program includes: Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; Quality information for the dataset the captured image, information indicating a position from which the sample image is cut out in the captured image, valid sample images included in the data set, and a valid image rate which is a ratio of the valid sample images to the total number of the sample images; The present invention causes a computer to execute a process of displaying, on a display device, shooting support information including the above. In still another aspect of the present disclosure, a program includes: Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; The computer is caused to execute a process of displaying on a display device shooting support information including quality information of the data set and an achievement level indicating the ratio of the number of valid sample images already obtained to the total number of images required for learning. Effect of the Invention

[0010] According to the present disclosure, even a person without any particular experience can capture images suitable for generating training data and create a high-quality training dataset. [Brief description of the drawings]

[0011] [Figure 1] 1 shows a schematic configuration of a data set creation system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a hardware configuration of a data set creation device according to a first embodiment. [Diagram 3] 1 is a block diagram showing a functional configuration of a data set creation device according to a first embodiment. FIG. [Figure 4]2 is a block diagram showing a configuration of a data quality estimation unit according to the first embodiment; FIG. [Diagram 5] 3 shows an example of a shooting support screen according to the first embodiment. [Figure 6] 13 is a flowchart of a display process of shooting support information. [Figure 7] FIG. 11 is a block diagram showing a configuration of a data quality estimation unit according to a second embodiment. [Figure 8] 13 shows an example of a shooting support screen according to the second embodiment. [Figure 9] FIG. 11 is a block diagram showing a functional configuration of a data set creation device according to a second embodiment. [Figure 10] 10 is a flowchart of a process performed by a data set creation device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings. First Embodiment [Overall configuration] 1 shows a schematic configuration of a data set creation system according to the first embodiment. The data set creation system 1 is a system that creates a learning data set used when training a recognition model that recognizes products from captured images.

[0013] As shown in the figure, the data set creation system 1 includes a data set creation device 100, a camera 2, and a display device 5. The camera 2 and the display device 5 are each connected to the data set creation device 100. The camera 2 is fixedly disposed at a predetermined position, such as on a shelf. An operator brings an object (product) for which a data set is to be created into the shooting range of the camera 2, and captures an image of the object with the camera 2. At this time, the operator rotates or changes the grip of the object to capture the appearance of the object from various angles. The camera 2 outputs captured images (video) of the object to the data set creation device 100.

[0014] The data set creation device 100 creates a data set for learning a target object based on a captured image input from the camera 2. The created data set is used to learn a recognition model that recognizes the object. By learning the recognition model using the created data set, the recognition model becomes able to recognize the object.

[0015] The data set creation device 100 generates shooting support information while an operator is photographing an object with the camera 2, and displays the information on the display device 5. The shooting support information is information that informs the operator of how the object is currently being photographed and the data set is being created, and if necessary, gives instructions or advice on how to photograph the object. The data set creation device 100 generates shooting support information based on the captured images captured by the camera 2 and analysis results such as the number and quality of data created based on the captured images, and outputs the information to the display device 5. Details of the shooting support information will be described later.

[0016] In this way, by displaying shooting support information while the worker is photographing an object and providing feedback to the worker about the current shooting status and data creation status, even workers with little experience can take images suitable for training a recognition model and efficiently create a high-quality data set.

[0017] [Hardware configuration] 2 is a block diagram showing a hardware configuration of the data set creation device 100 of the first embodiment. As shown in the figure, the data set creation device 100 includes an interface (I / F) 11, a processor 12, a memory 13, a recording medium 14, and a database (DB) 15.

[0018] The interface 11 inputs and outputs data to and from an external device. Specifically, the interface 11 acquires captured images from the camera 2 while the worker is taking a photograph, and outputs photographing support information to the display device 5. The interface 11 also outputs the created learning dataset to a learning device that learns a recognition model.

[0019] The processor 12 is a computer such as a CPU (Central Processing Unit) and executes a program prepared in advance to control the entire data set creation device 100. The processor 12 may be a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array). The processor 12 executes a display process of the shooting support information, which will be described later.

[0020] The memory 13 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 13 is also used as a working memory while the processor 12 is executing various processes.

[0021] The recording medium 14 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the data set creation device 100. The recording medium 14 records various programs executed by the processor 12. When the data set creation device 100 executes various processes, the programs recorded on the recording medium 14 are loaded into the memory 13 and executed by the processor 12. The DB 15 stores the captured images input from the camera 2 and the created data set.

[0022] [Function configuration] 3 is a block diagram showing the functional configuration of the data set creation device 100 of the first embodiment. The data set creation device 100 includes a captured image input unit 21, a data processing unit 22, a storage unit 23, a data quality estimation unit 24, and a display control unit 25. The captured image input unit 21 is configured by the interface 11, the data processing unit 22, the data quality estimation unit 24, and the display control unit 25 are mainly configured by the processor 12, and the storage unit 23 is configured by a DB 15.

[0023] The photographed image input unit 21 acquires photographed images of an object photographed by an operator from the camera 2, and outputs the images to the data processing unit 22. The photographed images are moving images of an object photographed continuously.

[0024] The data processing unit 22 uses the input photographed image to generate learning data for learning the recognition model. Specifically, the data processing unit 22 detects an object from the photographed image using an object detection model or the like, and cuts out an image of a part of the object to generate an image of the object (hereinafter referred to as a "sample image"). The object detection model detects an object from the photographed image, and outputs position information of a rectangle including the object, and a score indicating the likelihood of it being an object. The data processing unit 22 extracts the rectangular area detected by the object detection model from the photographed image to generate a sample image. Then, the data processing unit 22 outputs the generated sample image to the storage unit 23 together with the photographed image.

[0025] The storage unit 23 receives the captured image generated by the camera 2 and a plurality of sample images extracted from the captured image from the data processing unit 22, and stores them.

[0026] The data quality estimation unit 24 estimates the quality of the sample images generated by the data processing unit 22, selects sample images that satisfy a predetermined criterion as valid sample images, and stores them as learning data in the storage unit 23. In this way, a plurality of valid sample images that are determined to satisfy the criterion are collected, and a learning data set is created. Among the sample images generated by the data processing unit 22, those that do not satisfy the above criterion are not adopted as learning data and are discarded.

[0027] Furthermore, the data quality estimation unit 24 generates shooting support information based on the captured image and the estimated quality of the sample image, and outputs it to the display control unit 25. The shooting support information includes the captured image captured by the camera 2, the valid sample image, and the like, the details of which will be described later. The display control unit 25 creates a shooting support screen using the shooting support information input from the data quality estimation unit 24, and displays it on the display device 5.

[0028] [First Example] (Configuration of Data Quality Estimation Unit) Next, a first embodiment of the data set creation device 100 will be described. In the first embodiment, the data quality estimation unit 24 estimates the importance of the sample image in learning as the quality of the sample image. FIG. 4 is a block diagram showing the configuration of the data quality estimation unit 24a according to the first embodiment. As shown in the figure, the data quality estimation unit 24a includes an importance estimation unit 26.

[0029] The importance estimation unit 26 receives a sample image from the storage unit 23. The importance estimation unit 26 estimates the importance of the input sample image using an importance estimation model. In one example, an object detection model can be used as the importance estimation model. The object detection model outputs an object-likeness score of a detected object based on the input sample image, and the importance estimation unit 26 uses this score as the importance. The higher the object-likeness score, the higher the probability that the sample image contains an object, and the higher the suitability as learning data. Therefore, the data quality estimation unit 24a selects a sample image having an object-likeness score higher than a predetermined value as a valid sample image.

[0030] In another example, a model that estimates the image quality of an image can be used as the importance estimation model. As a model that estimates image quality, for example, a model that estimates the camera shake, blur, brightness, and the degree of obscuration of an object (the proportion of an object hidden behind other objects) of an image can be used. Specifically, based on the estimation result by the above model, the importance estimation unit 26 increases the importance as the camera shake, blur, appropriate brightness, and the degree of obscuration of an object become smaller.

[0031] If the image quality of a sample image is poor, specifically if there is a lot of camera shake, the image is blurred, the image is dark, most of the object is hidden, etc., the sample image is considered to have low suitability as learning data. Therefore, the data quality estimation unit 24a selects, as valid sample images, sample images whose image quality is determined to be equal to or higher than a reference level by the model for estimating image quality as described above.

[0032] Note that the importance may be calculated by combining the object-likeness score and the image quality. For example, the importance may be calculated by adding the object-likeness score and a value indicating the degree of camera shake, brightness, or the like, using a predetermined weight.

[0033] The data quality estimation unit 24a stores the valid sample images selected as described above in the storage unit 23. The set of valid sample images thus accumulated becomes a data set for learning.

[0034] Furthermore, the data quality estimation unit 24a outputs, as shooting support information, information such as the captured images input from the storage unit 23, the valid sample images selected by the importance estimation unit 26, and the reach and valid image rate calculated based on the number of valid sample images to the display control unit 25. The display control unit 25 generates a shooting support screen using the input shooting support information, and displays it on the display device 5.

[0035] (Shooting support screen) Fig. 5 shows an example of a photographing support screen according to the first embodiment. As shown in the figure, the photographing support screen is roughly divided into a photographed image display area 30 and an effective image display area 40. In the example of Fig. 5, the photographed image display area 30 displays a photographed image 31, a rectangle 32, an importance level 33, and an achievement level 34.

[0036] The captured image 31 is a real-time display of a captured image (moving image) captured by the camera 2. A rectangle 32 indicates the position of an object detected by an object detection model from the captured image 31. As described above, a portion of the rectangle 32 surrounding the object is cut out from the captured image 31 as a sample image. Importance 33 is the importance of the detected object, and more specifically, is the importance value calculated by the importance estimation unit 26 described above. Therefore, in the first embodiment, the importance value is the score of object-likeliness or an estimated value of the image quality of the sample image, as described above.

[0037] The achievement level 34 is the ratio of the number of valid sample images already obtained to the total number of images required for learning. The total number of images required for learning is determined in advance based on experience, etc. In the example of FIG. 5, it is shown that "120 (frames)" of valid sample images have been obtained at present, while the total number of images required for learning is "300 (frames)". This allows the worker to know how many sample images necessary for learning have been obtained, and how many more are still required. When the total number of valid sample images required have been obtained, the worker may be notified of this by display or sound.

[0038] On the other hand, the valid image display area 40 displays a thumbnail display area 41 and a valid image ratio 42. In the thumbnail display area 41, thumbnails of a plurality of valid sample images cut out from the photographed image 31 are displayed side by side. That is, as shown in the rectangle 32 in the photographed image display area 30, each thumbnail image 43 is a sample image that is determined to satisfy a predetermined standard by the above-mentioned data quality estimation unit 24a among the sample images cut out from the photographed image 31. Note that instead of displaying only the valid sample images as in FIG. 5, all sample images cut out from the photographed image 31 may be displayed in the thumbnail display area 41, and the valid sample images may be highlighted by changing the color of their frames, for example.

[0039] The valid image rate 42 is the ratio of valid sample images to the total number of sample images cut out from the captured image 31. In the example of FIG. 5, the valid image rate is 90%, which indicates that 90% of the sample images cut out from the captured image 31 to date have been adopted as valid sample images. In addition, the main factors determining whether an image is invalid, such as blurring 5%, lighting conditions 3%, and obscured objects 2%, may be displayed. If the valid image rate is lower than a predetermined standard, the operator may be notified of this by display or sound. This allows the operator to notice defects such as an object not being within the shooting range of the camera 2. Furthermore, if the valid image rate is lower than a predetermined standard, instructions or advice may be given to the operator. For example, if there is a lot of camera shake in the sample image and the valid image rate is low, a message such as "Please move a little more slowly" may be displayed or sound output may be output.

[0040] In the example of FIG. 5, the achievement level 34 and the valid image rate 42 are displayed as numerical values, but instead, they may be displayed as graphs, meters, or the like.

[0041] Furthermore, when photographing of all objects has been completed, the worker may be notified as to whether or not sufficient valid sample images have been acquired for all objects. For example, for objects for which valid sample images are insufficient due to an interruption in photographing, a message such as "Product X: 30 missing" may be displayed to notify the worker that the required number of sample images has not been reached. This allows the worker to take additional photographs of the missing sample images.

[0042] (Display processing) Next, the display processing of the shooting support information will be described. Fig. 6 is a flowchart of the display processing of the shooting support information. This processing is realized by the processor 12 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 3 and Fig. 4.

[0043] First, the captured image input unit 21 acquires a captured image from the camera 2 (step S11). Next, the data processing unit 22 cuts out a portion of an object from the captured image to generate a sample image (step S12). Next, the data quality estimation unit 24a estimates the importance of each sample image and extracts sample images that satisfy a predetermined criterion as valid sample images. In addition, the data quality estimation unit 24a calculates the achievement level and the valid image rate based on the number of valid sample images. Then, the data quality estimation unit 24a generates shooting support information including the captured image, the valid sample images, the achievement level, the valid image rate, and the like, and outputs the information to the display control unit 25 (step S13).

[0044] Next, the display control unit 25 generates a shooting support screen using the input shooting support information (step S14), and displays the shooting support screen on the display device 5 (step S15). In this way, the shooting support screen as exemplified in FIG. 5 is displayed on the display device 5.

[0045] Next, the data set creation device 100 determines whether or not to end the shooting (step S16). For example, when an instruction to end the shooting is input to the display device 5, or when the achievement degree reaches 100%, the data set creation device 100 determines to end the shooting. If it is determined not to end the shooting (step S16: No), the process returns to step S11, and steps S11 to S16 are repeated. On the other hand, if it is determined to end the shooting (step S16: Yes), the display process ends.

[0046] [Second Example] (Configuration of Data Quality Estimation Unit) Next, a second embodiment of the data set creation device 100 will be described. In the second embodiment, the data quality estimation unit 24 estimates existing object similarity as the quality of a sample image. FIG. 7 is a block diagram showing a configuration of a data quality estimation unit 24b according to the second embodiment. As shown in the figure, the data quality estimation unit 24b includes an existing object similarity estimation unit 27. The existing object similarity estimation unit 27 estimates the quality of the sample image by using an existing recognition model.

[0047] "Existing object similarity" refers to the similarity of a sample image to an existing object that has already been trained in an existing recognition model. In other words, existing object similarity indicates the degree to which a sample image is similar to other objects (products) that have already been registered in the recognition model. The similarity is calculated, for example, by using the cosine similarity for features extracted using a pre-trained feature extraction model. When training a recognition model to be able to recognize a new product A, the new product A may be similar to a product B that has already been registered in the recognition model. In this case, it is necessary to use a sample image of the new product A that is not similar to the existing product B, i.e., that is distinctive, for training the recognition model.

[0048] The data quality estimation unit 24b outputs, as photographing support information, to the display control unit 25, information such as the captured image input from the storage unit 23, the valid sample image selected by the existing object similarity estimation unit 27, and the reach and valid image rate calculated based on the number of valid sample images. In addition, for a sample image whose similarity to an existing object is higher than a predetermined standard, the existing object similarity estimation unit 27 outputs an image of the existing object determined to have a high similarity (hereinafter also referred to as a "similar object image") as photographing support information to the display control unit 25. The display control unit 25 generates a photographing support screen using the input photographing support information, and displays it on the display device 5.

[0049] (Shooting support screen) Fig. 8 shows an example of a photographing support screen according to the second embodiment. As in the first embodiment shown in Fig. 5, the photographing support screen includes a photographed image display area 30 and an effective image display area 40. The display contents of the effective image display area 40 are the same as in the first embodiment, so a description thereof will be omitted. As in the first embodiment, a photographed image 31, a rectangle 32, and a reach 34 are displayed in the photographed image display area 30.

[0050] Furthermore, in the second embodiment, when the existing object similarity estimation unit 27 determines that the similarity of the sample image to an existing object is higher than a predetermined standard, an image of the existing object, i.e., a similar object image 35, is displayed. In the example of Fig. 8, the similar object image 35 is displayed alongside a rectangle 32 corresponding to the sample image. Furthermore, a similarity 36 between the sample image and the similar object image calculated by the existing object similarity estimation unit 27 is displayed near the similar object image 35.

[0051] This allows the worker to see that there is an existing object similar to the currently created sample image, the image of the existing object, and the similarity to the existing object. Therefore, the worker can proactively photograph the surfaces or parts of the object currently being photographed that are highly distinguishable from existing objects with high similarity, making it possible to efficiently create learning data that is more distinguishable from existing objects. Specifically, the worker can photograph the product by changing the photographing location so that the displayed similarity value decreases. Note that, when there are multiple existing objects with high similarity, similar object images 35 may be displayed for a predetermined number of existing objects starting from the one with the highest similarity.

[0052] Also, the product name of the product indicated by the similar object image 35 may be displayed near the similar object image 35. Furthermore, when the similarity between the sample image and the existing object is higher than a predetermined value, the operator may be notified that the product currently being photographed is highly likely to have already been registered. This makes it possible to prevent duplicate registration of an object (product) that has already been registered in an existing recognition model.

[0053] (Display processing) Next, the display process of the photographing support information will be described. The display process of the photographing support information in the second embodiment is basically the same as that in the first embodiment shown in Fig. 6. However, in the second embodiment, in step S13, the existing object similarity estimation unit 27 generates photographing support information including the similar object image 35 and the similarity 36 with the existing object, and outputs it to the display control unit 25.

[0054] [Variations] Next, a modified example of the data set creation device 100 according to the first embodiment will be described. This modified example can be applied to the above-mentioned first or second embodiment. In the above-mentioned first embodiment, the data quality estimation unit 24a estimates the importance of a sample image, and adds a sample image whose importance meets a criterion to the data set as a valid sample image. At this time, in the modified example, the estimated importance is added to the valid sample image as attribute information. As a result, the sample images included in the data set stored in the storage unit 23 are in a state in which their importance is attached as attribute information. Therefore, when a recognition model is trained using the data set, the importance added to the sample image can be used to select sample images to be used for training and determine the priority. Specifically, by preferentially selecting sample images with high importance and training a recognition model, it is expected that the efficiency of training can be improved.

[0055] Similarly, in the second embodiment, the data quality estimation unit 24b estimates the existing object similarity of the sample image. At this time, in the modified example, the estimated existing object similarity is added to the valid sample image as attribute information and added to the data set. As a result, the sample images included in the data set stored in the storage unit 23 are in a state in which their existing object similarity is attached as attribute information. Therefore, when training a recognition model using the data set, the existing object similarity added to the sample image can be used to select sample images to be used for training and determine the priority. Specifically, by preferentially selecting sample images with low existing object similarity and training a recognition model, it is expected that the efficiency of training can be improved.

[0056] <Second embodiment> 9 is a block diagram showing the functional configuration of a data set creation device of the second embodiment. The data set creation device 70 includes an acquisition unit 71, a data processing unit 72, a data quality estimation unit 73, and a display control unit 74.

[0057] FIG. 10 is a flowchart of processing by the dataset creation device 70 of the second embodiment. First, the acquisition means 71 acquires a captured image of an object (step S21). Next, the data processing means 72 generates sample images, which are images of parts of the object, from the captured images, and generates a learning dataset including valid sample images that satisfy a predetermined standard among the sample images (step S22). Next, the data quality estimation means 73 estimates the quality of the dataset based on the captured image and the dataset (step S23). Then, the display control means 74 displays shooting support information including quality information of the dataset on the display device (step S24). Then, the processing ends.

[0058] According to the data set creation device 70 of the second embodiment, even a person without any particular experience can take images suitable for learning and create a high-quality learning data set.

[0059] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0060] (Appendix 1) An acquisition means for acquiring a photographed image of an object; a data processing means for generating sample images, which are images of portions of the object, from the captured images, and generating a learning dataset including valid sample images that satisfy a predetermined criterion among the sample images; a data quality estimation means for estimating a quality of the data set based on the captured image and the data set; a display control means for displaying, on a display device, imaging support information including quality information of the data set; A data set creation device comprising:

[0061] (Appendix 2) The dataset creation device according to claim 1, wherein the quality information includes an importance level of sample images included in the dataset.

[0062] (Appendix 3) The dataset creation device according to claim 2, wherein the importance includes a score of object-likeliness of the object detected from the sample image.

[0063] (Appendix 4) 4. The data set creation device according to claim 2, wherein the importance level includes information regarding image quality of the sample image.

[0064] (Appendix 5) The dataset creation device according to claim 1, wherein the quality information includes a similarity to an existing object as a result of recognizing a sample image included in the dataset using an existing recognition model.

[0065] (Appendix 6) The dataset creation device according to claim 1, wherein the quality information includes a similarity between a sample image included in the dataset and an existing object.

[0066] (Appendix 7) The shooting support information includes the captured image, information indicating a position from which the sample image is cut out in the captured image, valid sample images included in the dataset, and a valid image rate which is a ratio of valid sample images to a total number of the sample images.

[0067] (Appendix 8) The data set creation device according to any one of appendix 1 to 7, wherein the photographing support information includes an achievement level indicating a ratio of a number of valid sample images already obtained to a total number of images required for learning.

[0068] (Appendix 9) 9. The data set creation device according to any one of claims 1 to 8, further comprising a storage means for storing the data set by adding quality information estimated based on the valid sample images to the valid sample images.

[0069] (Appendix 10) Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; A data set creation method, comprising: displaying, on a display device, imaging support information including quality information of the data set.

[0070] (Appendix 11) Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; A recording medium having a program recorded thereon for causing a computer to execute a process of displaying, on a display device, imaging support information including quality information of the data set.

[0071] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-mentioned embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. [Explanation of symbols]

[0072] 1. Dataset creation system 12 processors 21 Photographed image input section 22 Data Processing Section 23 Memory section 24, 24a, 24b Data quality estimation section 25 Display control section 26 Importance Estimator 27 Existing object similarity estimation unit 100 Data set creation device

Claims

1. An acquisition means for acquiring a photographed image of an object; a data processing means for generating sample images, which are images of portions of the object, from the captured images, and generating a learning dataset including valid sample images that satisfy a predetermined criterion among the sample images; a data quality estimation means for estimating a quality of the data set based on the captured image and the data set; a display control means for displaying on a display device photographing support information including quality information of the data set, the photographed image, information indicating a position from which the sample image is cut out in the photographed image, valid sample images included in the data set, and a valid image rate which is a rate of the valid sample images to the total number of the sample images; A data set creation device comprising:

2. An acquisition means for acquiring a photographed image of an object; a data processing means for generating sample images, which are images of portions of the object, from the captured images, and generating a learning dataset including valid sample images that satisfy a predetermined criterion among the sample images; a data quality estimation means for estimating a quality of the data set based on the captured image and the data set; a display control means for displaying on a display device photographing support information including quality information of the data set and an achievement level indicating a ratio of the number of valid sample images already obtained to the total number of images required for learning; A data set creation device comprising:

3. The data set creation device according to claim 1 , wherein the quality information includes an importance level of a sample image included in the data set.

4. The data set creation device according to claim 3 , wherein the importance includes a score of object-likeliness of the object detected from the sample image.

5. The data set creation device according to claim 3 , wherein the importance includes information regarding image quality of the sample image.

6. The data set creation device according to claim 1 or 2, wherein the quality information includes a similarity to an existing object as a result of recognizing a sample image included in the data set using an existing recognition model.

7. 1. A computer implemented method for creating a data set, comprising: Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; A dataset creation method that displays on a display device shooting support information including quality information of the dataset, the captured image, information indicating the position from which the sample image is cut out in the captured image, valid sample images included in the dataset, and a valid image rate which is the ratio of valid sample images to the total number of sample images.

8. A computer-implemented method for creating a dataset, comprising: Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; A dataset creation method that displays, on a display device, shooting support information including quality information of the dataset and an achievement level indicating the ratio of the number of valid sample images already obtained to the total number of images required for learning.

9. Obtaining a photographic image of the object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; A program that causes a computer to execute a process of displaying on a display device shooting support information including quality information of the dataset, the captured image, information indicating the position from which the sample image is cut out in the captured image, valid sample images included in the dataset, and a valid image rate which is the ratio of valid sample images to the total number of sample images.

10. Obtaining a photographed image of an object; generating sample images that are images of portions of the object from the captured images, and generating a learning dataset that includes valid sample images that satisfy a predetermined criterion among the sample images; Estimating a quality of the dataset based on the captured images and the dataset; A program that causes a computer to execute a process of displaying, on a display device, shooting support information including quality information of the dataset and an achievement level indicating the ratio of the number of valid sample images already obtained to the total number of images required for learning.

Images