Recognition model generation method and recognition model generation device

Abstract

To provide a recognition model generation method and a recognition model generation device.SOLUTION: In a recognition model generation apparatus, a control unit includes first recognition model generating means, providing means, and second recognition model generating means. The first recognition model generating means generates a first recognition model based on a plurality of composite images. The first recognition model outputs an object recognition result for input of an image. The providing means provides the object recognition result, by input of a plurality of captured images of a detection target into the first recognition model, to the captured images as annotation data. The second recognition model generating means generates, based on the captured image of the detection target and the annotation data, a second learning model.SELECTED DRAWING: Figure 2

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Patent Application No. 2021-117345, filed in Japan on July 15, 2021, the entire disclosure of which is incorporated herein by reference. [Technical field]

[0002] The present disclosure relates to a recognition model generation method and a recognition model generation device. [Background technology]

[0003] Conventionally, machine learning-based image recognition has been known as a method for recognizing a specific object from within an entire image. For example, an image recognition method has been proposed that uses learning images generated from CAD data to perform image recognition (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-191973 A Summary of the Invention

[0005] The method for generating a recognition model according to the first aspect includes the steps of: obtaining a plurality of composite images showing the detection target; A first learning is performed to create a first recognition model that outputs an object recognition result for an input of an image based on the plurality of synthetic images; acquiring a captured image of the detection target; adding the object recognition results output by inputting the plurality of captured images into the first recognition model as annotation data to the captured images; A second learning is performed to create a second recognition model based on the captured image and the annotation data.

[0006] Further, a recognition model generating device according to a second aspect of the present invention comprises: a first recognition model generating means for generating a first recognition model that outputs an object recognition result for an input of an image, based on a plurality of composite images showing a detection target; an annotation means for inputting a plurality of captured images of the detection target to the first recognition model, and adding the object recognition result to the captured images as annotation data; and a second recognition model generating means for generating a second recognition model based on the captured image and the annotation data.

[0007] A recognition model generating device according to a third aspect comprises: A recognition model generation device that generates a second recognition model by training a first recognition model using a captured image of a detection target as training data, The first recognition model is a recognition model generated by training an original recognition model used for object recognition using a synthetic image generated based on three-dimensional shape data of a detection target as training data. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a functional block diagram showing a schematic configuration of a recognition model generating device according to an embodiment. [Diagram 2] 2 is a functional block diagram showing a virtual schematic configuration of a control unit in FIG. 1. [Diagram 3] 4 is a first flowchart for explaining a recognition model generation process executed by a control unit in FIG. [Figure 4] 10 is a second flowchart for explaining the recognition model generation process executed by the control unit in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In order to improve the recognition accuracy by machine learning, it is necessary to learn using a large amount of training data. For example, a large amount of training data requires images of the same object to be recognized from various directions and images of the same object to be recognized under various lighting conditions. In order to prepare a large amount of training data for such an object to be recognized, it is known to generate training images from CAD data of the object. It is difficult for a recognition model trained only using training images generated from CAD data to accurately recognize real captured images.

[0010] Hereinafter, an embodiment of a recognition model generation device to which the present disclosure is applied will be described with reference to the drawings. Note that a recognition model generation method will be described in conjunction with an explanation of the recognition model generation device.

[0011] The recognition model generation device creates a first recognition model by training an original recognition model using a synthetic image based on three-dimensional shape data of the detection target. The recognition model generation device uses the first recognition model to annotate at least a part of a captured image of the detection target, thereby adding annotation information. The recognition model generation device creates a deployment model via a second recognition model by training the first recognition model. The recognition model generation device uses the captured image of the detection target to which annotation data has been added, to create the deployment model.

[0012] 1, a recognition model generation device 10 according to an embodiment may include a communication unit 11, a storage unit 12, and a control unit 13. The recognition model generation device 10 is, for example, one or more server devices capable of communicating with each other, a general-purpose electronic device such as a PC (Personal Computer), or a dedicated electronic device.

[0013] The communication unit 11 may communicate with an external device. The external device may be, for example, an imaging device, a storage medium, or a terminal device. The imaging device may be provided in, for example, a mobile terminal such as a smartphone or a tablet, or a device such as a robot. The storage medium may be, for example, any storage medium that is detachable via a connector. The terminal device may be, for example, a general-purpose electronic device such as a smartphone, a tablet, or a PC, or a dedicated electronic device. The communication unit 11 may communicate with the external device in a wired or wireless manner.

[0014] The communication unit 11 may obtain information and instructions through communication with an external device. The communication unit 11 may provide information and instructions through communication with an external device.

[0015] The communication unit 11 may acquire three-dimensional shape data of the detection target. The three-dimensional shape data is, for example, CAD data. The name of the detection target may be associated with the three-dimensional shape data as label data.

[0016] The communication unit 11 may acquire texture information of the detection target. As the texture data, the texture of a material that is generally used for the expected detection target may be digitized as a template, or the surface of a photograph of the actual object may be digitized.

[0017] The communication unit 11 may acquire a composite image generated based on three-dimensional shape data of the detection target. The acquired composite image may be associated with annotation data. The annotation data may include data corresponding to at least one of a mask image of the detection target, a bounding box of the detection target, and a label, for example. The mask image is, for example, an image that fills in the contour of the detection target within the entire image range. The bounding box is, for example, a rectangular frame that surrounds the detection target. The label is, for example, the name of the detection target. Note that the composite image may be generated based on, for example, multiple two-dimensional shape data.

[0018] The communication section 11 may acquire a captured image of the detection target. As described later, the communication section 11 may acquire annotation data that has been corrected with respect to annotation data that is to be added to the captured image.

[0019] The communication unit 11 may provide an imaging guide for imaging the detection target to the mobile terminal or the robot, as described later. The communication unit 11 may provide annotation information obtained by using a first recognition model for the acquired captured image, to the terminal device, as described later.

[0020] The storage unit 12 includes any storage device, such as a random access memory (RAM) and a read only memory (ROM). The storage unit 12 may store various programs that cause the control unit 13 to function and various information that the control unit 13 uses.

[0021] The control unit 13 includes one or more processors and memories. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The control unit 13 may be either a system-on-a-chip (SoC) or a system in a package (SiP) in which one or more processors work together.

[0022] As shown in FIG. 2, the control unit 13 may function as a synthesis means 14, a first recognition model generation means 15, an imaging guide generation means 16, an assignment means 17, and a second recognition model generation means 18, which will be described below.

[0023] When the synthesis means 14 acquires three-dimensional shape data of the detection target via the communication unit 11, the synthesis means 14 may generate a composite image of the detection target based on the three-dimensional shape data. The synthesis means 14 may generate a two-dimensional composite image including an image of a single or multiple detection targets in an image display area such as a rectangle based on the three-dimensional shape data. The synthesis means 14 may generate multiple composite images. The synthesis means 14 may generate a composite image in which the images of the detection targets are arranged in a variety of ways in the image display area. The synthesis means 14 may generate a composite image including images of different detection targets separately. The synthesis means 14 may generate a composite image including different detection targets. The synthesis means 14 may generate a composite image so as to have a format of input information to be input when inferring the first recognition model described later. For example, if the captured image input to the first recognition model is two-dimensional, the composite image may also be two-dimensional.

[0024] The synthesis means 14 may generate a synthetic image including images of various postures of the detection target in the image display area. The synthesis means 14 may determine the posture of the image based on the three-dimensional shape data of the detection target. For example, when the detection target is spherical, the synthesis means 14 generates a synthetic image in which an image seen from an arbitrary direction is the posture of the detection target. For example, when the detection target is cubic, the synthesis means 14 may generate an angular image seen from a direction tilted 45° from an arbitrary face around an arbitrary side as an axis and rotated 10° around an edge perpendicular to the arbitrary side as an axis as a synthetic image. The synthesis means 14 may further generate an angular image seen from a direction tilted 50° from an arbitrary face around an arbitrary side as an axis and rotated 10° around an edge perpendicular to the arbitrary side as an axis as a synthetic image.

[0025] When multiple composite images are generated for the same detection target, the synthesis means 14 may determine that some of the images are used as learning data and the other parts are used as evaluation data. For example, when a composite image of a cubic detection target is generated as described above, a composite image viewed from an arbitrary face in a direction tilted at 45° around an arbitrary side may be determined as learning data. Also, a composite image viewed from an arbitrary face in a direction tilted at 50° around an arbitrary side may be determined as evaluation data. Furthermore, the learning data may be determined as training data or validation data.

[0026] The synthesis means 14 may generate a synthetic image using a texture corresponding to the detection target. The texture corresponding to the detection target may be selected by specifying a template registered in advance for each type of material, such as metal, and stored in the storage unit 12, or an image of the material. The image of the material may be an image of a texture corresponding to a material identified based on an overall image generated by capturing an image of the detection target by an imaging means, such as a camera. The image of the material may be stored in advance in the storage unit 12. The texture may be selected by detecting manual input to a pointing device, such as a mouse, or an input device, such as a keyboard, via the communication unit 11.

[0027] When the synthesis means 14 receives a command to create a synthetic image from the assignment means 17, which will be described later, the synthesis means 14 may generate a synthetic image based on the three-dimensional shape data so as to reproduce the characteristics of the captured image. In other words, the synthesis means 14 may generate a synthetic image so as to have the same characteristics as the captured image. The same characteristics are, for example, the same posture as the detection target in the captured image, in other words, the same appearance, and the same color, in other words, the same hue, saturation, and brightness. The synthesis means 14 may store the newly generated synthetic image in the storage unit 12 as data for creating a deployment model, which will be described later.

[0028] The synthesis means 14 may annotate the composite image based on the three-dimensional shape data. Annotation refers to adding annotation data to the composite image. That is, the synthesis means 14 may add annotation data to the composite image by performing annotation. The annotation data added by the synthesis means 14 through annotation may include, for example, a mask image of the detection target and a bounding box of the detection target. The synthesis means 14 may generate a polygon based on the three-dimensional shape data, and generate a mask image and a bounding box surrounding the polygon by calculating an area occupied by the detection target as viewed from the shooting direction of the composite image. The synthesis means 14 may store the composite image to which the annotation data has been added in the storage unit 12 as data for creating a deployment model.

[0029] The first recognition model generating means 15 performs a first learning process to train an original recognition model using a synthetic image as training data. The original recognition model is a recognition model used for object recognition. The original recognition model is a model that detects an area for each object using at least one of a mask image and a bounding box in the shape of a rectangular frame in order to perform object detection such as instance segmentation. The original recognition model may be a trained model using a large amount of data set such as ImageNet or MS COCO, or a data set of a specific product group such as industrial products. The first learning process is, for example, transfer learning and fine tuning of the original recognition model. The first recognition model generating means 15 generates a first recognition model by the first learning process. The first recognition model outputs an object recognition result for any input image. The object recognition result may be data corresponding to at least one of a mask image of the detection target, a bounding box of the detection target, a label, a mask score, and a bounding box score.

[0030] The first recognition model generation means 15 may calculate the accuracy for the validation data for each epoch in learning using the training data. The first recognition model generation means 15 may attenuate the learning rate if the accuracy for the validation data does not increase a certain number of times. Furthermore, the first recognition model generation means 15 may end learning if the accuracy for the validation data does not increase a certain number of times.

[0031] The first recognition model generation means 15 may store the model of the epoch having the best accuracy for the validation data as the first recognition model in the storage unit 12. The first recognition model generation means 15 may search for a confidence threshold that provides the best accuracy for the validation data while changing the confidence threshold. The first recognition model generation means 15 may determine the searched confidence threshold as the confidence threshold of the first recognition model. The first recognition model generation means 15 may evaluate the first recognition model using the evaluation data.

[0032] The imaging guide generating means 16 may provide an imaging guide based on the acquired three-dimensional shape data. The imaging guide may indicate an imaging method of the detection target corresponding to the acquired three-dimensional shape data. The imaging guide may include, for example, a designation of the imaging direction of the detection target, in other words, how the detection target appears in the captured image generated by imaging. The imaging guide may include, for example, a designation of the size of the image of the detection target in the entire captured image, in other words, the focal length and the distance between the detection target and the camera. The imaging guide generating means 16 may determine the imaging direction and image size of the detection target based on the three-dimensional shape data.

[0033] The imaging guide may be sent to a mobile terminal with an imaging device, such as a smartphone or a tablet, or to a control device of a robot equipped with an imaging device. The imaging device may capture images under control based on the imaging guide, and obtain an image of the detection target.

[0034] In a configuration in which the imaging guide is sent to a mobile terminal, the imaging guide may be an imaging method illustrated by text and drawings. The detection target may be imaged by manual operation of a user referring to the imaging guide. In a configuration in which the imaging guide is sent to a control device of a robot, the imaging guide may be a control command for causing the robot to adjust the position of the imaging device so as to achieve a specified imaging direction and size. The detection target may be imaged at a position adjusted by the robot based on the imaging guide.

[0035] The control unit 13 may acquire the captured image via the communication unit 11. When acquiring the captured image, the control unit 13 may present a name of the detection target corresponding to the acquired three-dimensional shape data in a selectable manner. The control unit 13 may present the name of the detection target, for example, on a display connected to the recognition model generating device 10 or a terminal device. The control unit 13 may acquire the name corresponding to the captured image by an operation input from an input device connected to the recognition model generating device 10 or a terminal device. The control unit 13 may associate the name of the detection target as a label with the acquired captured image.

[0036] The adding means 17 adds annotation data to the captured image by annotating at least a part of the acquired captured image using the first recognition model. The annotation data may include data corresponding to at least one of a mask image of the detection target and a bounding box of the detection target. The adding means 17 may store the captured image to which the annotation data has been added in the storage unit 12 as data for creating a deployment model.

[0037] The adding means 17 may generate a removed image by performing noise removal on the captured image to be annotated. In the annotation, the adding means 17 may perform annotation by having the removed image recognized by the first recognition model, and may add annotation data to the captured image corresponding to the removed image. Therefore, the generated removed image is not used by the second recognition model generating means 18 described later, and the second learning is performed using the captured image to which the annotation data has been added.

[0038] The assigning means 17 may present the captured image to which the annotation data has been assigned, on a display connected to the recognition model generating device 10 or on a terminal device connected via the communication unit 11. The annotation data may be correctable by an operation input to an input device or the terminal device connected to the recognition model generating device 10. The assigning means 17 may acquire the corrected annotation data via the communication unit 11. The assigning means 17 may use the corrected annotation data to update the annotation data stored in the storage unit 12 as data for creating a deployment model.

[0039] If the confidence in the annotation of the captured image, i.e., the confidence when the captured image is recognized by the first recognition model in order to annotate it, is below a threshold, the assignment means 17 may assign an instruction to the synthesis means 14 to create a synthetic image using the features of the captured image.

[0040] The second recognition model generation means 18 performs second learning to train the first recognition model using the captured image. The second recognition model generation means 18 generates a second recognition model by the second learning. The second recognition model outputs an object recognition result for any input image. The object recognition result may be data corresponding to at least one of a mask image of the detection target, a bounding box of the detection target, a label, a mask score, and a bounding box score.

[0041] The second recognition model generating means 18 may generate the second recognition model by performing a second learning using the captured image to which the annotation data is added as teacher data. The second recognition model generating means 18 may perform the second learning using a composite image to which the annotation data is added, which is stored in the storage unit 12 as data for creating a deployment model.

[0042] In a configuration in which the second learning is performed using captured images to which annotation data has been added, the second recognition model generation means 18 may determine, as learning data, at least a portion of the captured images to which annotation data has been added and which are stored in the storage unit 12 as data for creating a deployment model. Furthermore, the second recognition model generation means 18 may determine the learning data as training data or validation data. The second recognition model generation means 18 may determine, as evaluation data, another portion of the captured images to which annotation data has been added.

[0043] The second recognition model generation means 18 may calculate the accuracy for the validation data for each epoch in learning using the training data. The second recognition model generation means 18 may attenuate the learning rate if the accuracy for the validation data does not increase a certain number of times. Furthermore, the second recognition model generation means 18 may end learning if the accuracy for the validation data does not increase a certain number of times.

[0044] The second recognition model generation means 18 may store the model of the epoch having the best accuracy for the validation data in the storage unit 12 as the second recognition model. The second recognition model generation means 18 may search for a confidence threshold that provides the best accuracy for the validation data while changing the confidence threshold. The second recognition model generation means 18 may determine the searched confidence threshold as the confidence threshold of the second recognition model. The second recognition model generation means 18 may evaluate the second recognition model using the evaluation data.

[0045] The second recognition model generation means 18 may generate the second recognition model by performing re-learning of the first recognition model by performing domain adaptation using captured images to which no annotation data has been added, as a second learning step.

[0046] In a configuration in which the second learning is performed using captured images to which no annotation data is added, the second recognition model generation means 18 may determine, as evaluation data, at least a part of the captured images to which annotation data is added and which are stored in the storage unit 12 as data for creating a deployment model. The second recognition model generation means 18 may evaluate the second recognition model using the evaluation data.

[0047] The second recognition model generation means 18 may store the evaluated second recognition model in the storage unit 12 as a deployment model.

[0048] Next, the recognition model generation process executed by the control unit 13 in this embodiment will be described with reference to the flowcharts in Figures 3 and 4. The recognition model generation process starts, for example, when an operation input for starting the generation process is detected to an input device or the like connected to the recognition model generation device 10.

[0049] In step S100, the control unit 13 determines whether or not the three-dimensional shape data of the detection target has been acquired. If not, the process returns to step S100. If acquired, the process proceeds to step S101.

[0050] In step S101, the control unit 13 generates a synthetic image based on the three-dimensional shape data whose acquisition has been confirmed in step S100. After generation, the process proceeds to step S102.

[0051] In step S102, the control unit 13 generates annotation data based on the three-dimensional shape data whose acquisition has been confirmed in step S100. The control unit 13 assigns the generated annotation data to the composite image generated in step S101. After the assignment, the process proceeds to step S103.

[0052] In step S103, the control unit 13 executes a first learning by training an original recognition model using the synthetic image to which the annotation data has been added in step S102. The control unit 13 stores the first recognition model generated by executing the first learning in the storage unit 12. After executing the first learning, the process proceeds to step S104.

[0053] In step S104, the control unit 13 may generate an imaging guide based on the three-dimensional shape data confirmed to be acquired in step S100. The control unit 13 may generate an imaging guide according to the destination. After generation, the process proceeds to step S105.

[0054] In step S105, the control unit 13 provides the imaging guide generated in step S104 to the external device. After providing, the process proceeds to step S106.

[0055] In step S106, the control unit 13 determines whether or not a captured image is acquired from an external device. If a captured image is not acquired, the process returns to step S106. If a captured image is acquired, the process proceeds to step S107.

[0056] In step S107, the control unit 13 presents, in a selectable manner, the name of the detection target corresponding to the three-dimensional shape data stored in the storage unit 12. After the presentation, the process proceeds to step S108.

[0057] In step S108, the control unit 13 determines whether or not the name of the detection target has been acquired. If the name of the detection target has been acquired, the process proceeds to step S109. If the name of the detection target has not been acquired, the process proceeds to step S110.

[0058] In step S109, the control unit 13 associates the name confirmed to have been acquired in step S108 with the captured image confirmed to have been acquired in step S106. The control unit 13 stores the captured image associated with the name of the detection target in the storage unit 12. After the association, the process proceeds to step S110.

[0059] In step S110, the control unit 13 removes noise from the captured image whose acquisition was confirmed in step S106 to generate a removed image. After noise removal, the process proceeds to step S111.

[0060] In step S111, the control unit 13 uses the first recognition model generated in step S103 to annotate the removed image generated in step S110. The control unit 13 assigns the annotation data generated by the annotation to the captured image corresponding to the removed image. After the annotation, the process proceeds to step S112.

[0061] In step S112, the control unit 13 presents the captured image to which the annotation data has been added. After the presentation, the process proceeds to step S113.

[0062] In step S113, the control unit 13 determines whether or not the corrected annotation data is acquired for the presentation of the captured image to which the annotation data is added. If the corrected annotation data is acquired, the process proceeds to step S114. If the corrected annotation data is not acquired, the process proceeds to step S115.

[0063] In step S114, the control unit 13 uses the corrected annotation data, the acquisition of which was confirmed in step S113, to update the annotation data stored in the storage unit 12. After updating, the process proceeds to step S115.

[0064] In step S115, the control unit 13 generates a second recognition model by executing the second learning. In a configuration in which a captured image to which annotation data is added is used in the second learning, the control unit 13 generates a composite image having the same features as a captured image in which the certainty in the annotation is equal to or lower than a threshold. The control unit 13 further trains the first recognition model using a captured image to which annotation data is added and a newly generated composite image. Alternatively, in a configuration in which a captured image to which no annotation data is added is used in the second learning, the control unit 13 performs domain adaptation using the captured image. After the second learning is executed, the process proceeds to step S116.

[0065] In step S116, the control unit 13 evaluates the second recognition model generated in step S116 by using the captured image to which the annotation data has been added. After the evaluation, the process proceeds to step S117.

[0066] In step S117, the control unit 13 stores the second recognition model evaluated in step S116 as a deployment model in the storage unit 12. After storage, the recognition model generation process ends.

[0067] The recognition model generating device 10 of the present embodiment configured as described above generates a first recognition model that outputs an object recognition result in response to an image input based on a plurality of composite images showing a detection target, and uses the object recognition results to add annotation data to the captured image by inputting a plurality of captured images of the detection target to the first recognition model, and creates a second learning model based on the captured image and the annotation data. With this configuration, the recognition model generating device 10 annotates the captured image using the first recognition model, so that it is possible to reduce the annotation work of the captured image for learning the recognition model of the detection target using the composite images and the captured images.

[0068] In addition, since the recognition model generating device 10 creates the second learning model as described above, it is possible to improve the recognition accuracy of the detection target in the actually captured image. In addition, since the recognition model generating device 10 can learn using a large number of synthetic images generated based on three-dimensional shape data, it is possible to generate a model with high recognition accuracy even if the number of captured images is small.

[0069] When the actual object to be detected is manufactured on a manufacturing line, the actual object is manufactured using three-dimensional shape data. Therefore, the three-dimensional shape data of the object to be detected is generated before the captured image of the object to be detected is prepared, so that the composite image can be obtained before the captured image. In the recognition model generation method of the present embodiment configured as described above, until the actual object to be detected is manufactured and the captured image can be acquired, the first recognition model can be created by learning the original recognition model using the composite image. After that, after the actual object to be detected is manufactured and the captured image can be acquired, annotation data can be added to at least a part of the captured image using the first recognition model, and the first recognition model can be learned using the captured image of the object to be detected, so that the recognition model generation method of the present embodiment can proceed with the construction of the manufacturing line and the generation of the recognition model in parallel, so that a manufacturing line in which recognition model generation is introduced can be introduced at an early stage.

[0070] Furthermore, in the second learning, the recognition model generating device 10 of this embodiment generates a second recognition model using a captured image to which annotation data has been added. With this configuration, the recognition model generating device 10 can reduce the time required for the second learning.

[0071] In the second learning, the recognition model generating device 10 of this embodiment re-learns the first recognition model by performing domain adaptation using a captured image of the detection target to which no annotation data is added, and the captured image to which the annotation data is added is used to evaluate the second recognition model. With this configuration, the recognition model generating device 10 evaluates the learned recognition model using a captured image instead of a synthetic image, thereby improving the reliability of the evaluation result.

[0072] Furthermore, when the confidence level in the annotation of the captured image, i.e., the confidence level when the first recognition model is made to recognize the captured image to annotate it, is equal to or lower than a threshold value, the recognition model generating device 10 of this embodiment generates a synthetic image of the detection target so that the target has the same features as the captured image, and uses the synthetic image for the second learning. With this configuration, the recognition model generating device 10 can generate a large number of synthetic images that look similar to the appearance that reduces the recognition accuracy, thereby improving the recognition accuracy of the finally learned second recognition model. Furthermore, with the above-mentioned configuration, the recognition model generating device 10 can improve the recognition accuracy of the detection target in the actual captured image by using the captured image while ensuring robustness in the domain of the synthetic image.

[0073] Moreover, the recognition model generating device 10 of the present embodiment provides an imaging guide based on three-dimensional shape data. With such a configuration, the recognition model generating device 10 can capture an image based on the imaging guide. Therefore, the recognition model generating device 10 can obtain a captured image capturing an image of a detection target in a posture that is highly necessary to learn, based on three-dimensional shape data, regardless of the user's experience and knowledge. As a result, the recognition model generating device 10 can finally generate a second recognition model with high recognition accuracy.

[0074] Furthermore, the recognition model generating device 10 of this embodiment assigns annotation data by having the first recognition model recognize the image obtained by removing noise from the captured image in the annotation, and trains the first recognition model using the captured image in the second learning. With this configuration, the recognition model generating device 10 can assign highly accurate annotation data by making the captured image closer to a synthetic image with less noise in the annotation. Furthermore, the recognition model generating device 10 performs learning in the second learning by using the captured image without removing noise as it is, and therefore can improve the recognition accuracy of the detection target in the actual captured image.

[0075] Furthermore, the recognition model generating device 10 of this embodiment generates a synthetic image using texture. With this configuration, the recognition model generating device 10 can further improve the recognition accuracy of the first recognition model and the second recognition model.

[0076] The above describes an embodiment of the recognition model generating device 10. However, the embodiment of the present disclosure can also be implemented as a method or program for implementing the device, or as a storage medium on which a program is recorded (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card, etc.).

[0077] The form of implementation of the program is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter, but may be a program module incorporated into an operating system, etc. Furthermore, the program may or may not be configured so that all processing is performed only by the CPU on the control board. The program may be configured so that a part or all of it is executed by another processing unit mounted on an expansion board or expansion unit added to the board as necessary.

[0078] The drawings for explaining the embodiments according to the present disclosure are schematic, and the dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones.

[0079] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions, etc. included in each component, etc. can be rearranged so as not to cause logical inconsistencies, and multiple components, etc. can be combined into one or divided.

[0080] All of the features described in this disclosure and / or all steps of any method or process disclosed herein may be combined in any combination except those combinations in which the features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is merely one example of a generic series of the same or equivalent features.

[0081] Furthermore, the embodiments of the present disclosure are not limited to any of the specific configurations of the embodiments described above, but rather extend to any novel feature or combination of features described herein, or any novel method or process step or combination of features described herein.

[0082] In this disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configurations. Configurations distinguished by descriptions such as "first" and "second" in this disclosure may exchange numbers in the configurations. For example, the first recognition model may exchange identifiers "first" and "second" with the second recognition model. The exchange of identifiers is performed simultaneously. The configurations remain distinguished even after the exchange of identifiers. Identifiers may be deleted. A configuration from which an identifier has been deleted is distinguished by a code. Descriptions of identifiers such as "first" and "second" in this disclosure alone should not be used to interpret the order of the configurations or to justify the existence of identifiers with smaller numbers.

[0083] In the present disclosure, the synthesis means 14, the first recognition model generation means 15, the imaging guide generation means 16, the assignment means 17, and the second recognition model generation means 18 have been described as being functioned by the control unit 13, but this is not limited thereto. The synthesis means 14, the first recognition model generation means 15, the imaging guide generation means 16, the assignment means 17, and the second recognition model generation means 18 may each be configured by one or more devices. In this case, the recognition model generation method disclosed in the present embodiment may be implemented in a recognition model generation system including, for example, a synthesis device, a first recognition model generation device, an imaging guide generation device, an assignment device that assigns annotation data, and a second recognition model generation device. [Explanation of symbols]

[0084] 10. Recognition model generation device 11 Communications Department 12 Storage section 13 Control section 14 Synthesis means 15 First recognition model generating means 16 Imaging guide generation means 17 Means of Granting 18 Second recognition model generating means

Claims

1. A recognition model for performing object recognition, and obtaining an image of a detection target, updating the recognition model based on the image and the object recognition result output by inputting the image into the recognition model. A recognition model generation method.

2. The method for generating a recognition model according to Claim 1, wherein the update of the recognition model is performed by assigning an object recognition result to at least a part of the image.

3. The method for generating a recognition model according to Claim 1, wherein the object recognition result is either a mask image which is an image obtained by filling the inside of the contour of the detection target or a bounding box which is a rectangle frame surrounding the detection target.

4. As the image, obtaining a plurality of composite images, updating the recognition model based on the object recognition result and the composite image. The method for generating a recognition model according to Claim 1.

5. As the image, obtaining a removed image obtained by removing noise from a captured image of a detection target captured by an imaging device, updating the recognition model by assigning an object recognition result to the removed image. The method for generating a recognition model according to Claim 1.

6. Further obtaining three-dimensional shape data of the detection target, determining the shooting direction and the size of the image of the detection target based on the three-dimensional shape data, and obtaining an image of the detection target. The method for generating a recognition model according to Claim 1.

7. The three-dimensional shape data has the name of the detection target associated as a label, and the name of the detection target is assigned as the object recognition result to the image generated based on the three-dimensional shape data. The method for generating a recognition model according to Claim 1.

8. A recognition model for performing object recognition, and an acquisition means for obtaining an image of a detection target, and an update means for updating the recognition model based on the image and the object recognition result output by inputting the image into the recognition model. A recognition model generation device.

9. The recognition model generation device according to Claim 8, and a robot that performs object recognition using the recognition model updated by the recognition model generation device and executes a predetermined operation. A robot system comprising the same.

Images