Object recognition device and computer-readable recording medium

Abstract

An object recognition device according to the present disclosure comprises: an image acquisition unit that acquires a first image obtained by taking a picture of an object; a feature extraction unit that extracts image features from the first image; an area classification unit in which comparison features, which indicate features of an image area to be subjected to comparison processing, and an exclusion feature, which is a feature of an image area to be excluded from comparison processing, are used by a classifier as a basis for classifying areas in which the comparison features appear among the image features and areas in which the exclusion feature appears, in order to classify an area in which the exclusion feature appears among the image features extracted by the feature extraction unit as an exclusion area;a feature exclusion unit that creates target features obtained by excluding image features within the exclusion area from the image features extracted by the feature extraction unit; a feature comparison unit that performs comparison processing between the target features and the comparison features; and an output unit that outputs the result of the comparison processing by the feature comparison unit.

Description

field of technology

[0001] The present disclosure relates to an object recognition device and a computer-readable recording medium. State of the art

[0002] When a robot performs work on an object, such as a workpiece, whose position and orientation are unknown, the object's position and orientation are sometimes specified using an image (for example, patent literature 1 and the like). For instance, an image feature representing a specific area of ​​an object is extracted from an image obtained by capturing the object, whose position and orientation are known, and registered along with the position and orientation.Similarly, for an object whose position and orientation are unknown, an image feature representing a specific area of ​​the object is extracted from the acquired image in the same way. The object's position and orientation are then specified from the change in position and orientation on the image, obtained by collating it with the recorded image feature (model feature) and the recorded position and orientation. The image features used for collation include an edge feature representing an area with a large luminance difference, a corner feature representing the intersection of two or more edges, and so on. List of quotations Patent literature

[0003] Patent Literature 1: JP 2002-319022 A Overview of the invention Technical task

[0004] If an image feature (model feature) used for collation is not prominent in an image, the threshold used to determine whether a feature should be extracted is set low, resulting in the extraction of a large number of image features in addition to the target area. Consequently, an area is created that is incorrect but closely matches the model feature, potentially leading to the false detection of an object. Furthermore, this also increases the processing time required for image feature collation.

[0005] Therefore, a technique to improve the recognition accuracy of an object in an image was desired. Technical solution

[0006] An object recognition device according to the present disclosure solves the above problem by performing learning such that image features extracted by feature extraction processing can be classified into a collation feature, which indicates a feature of an object, and an exclusion feature, which indicates an unnecessary feature. During object recognition, the captured image features are classified into the collation feature and the exclusion feature, and after the image feature classified as an exclusion feature has been excluded, collation processing is performed.

[0007] One aspect of the present disclosure is an object recognition device comprising: an image capture unit that captures a first image obtained by picking up an object; a feature extraction unit that extracts image features from the first image; an area classification unit that classifies as an exclusion area an area in which an exclusion feature appears among the image features extracted by the feature extraction unit, by means of a classifier that classifies an area in which a collation feature appears and the area in which the exclusion feature appears, based on the collation feature, which indicates a feature of an image area to be a target of collation processing, and the exclusion feature, which is a feature of an image area to be excluded from the target of collation processing;a feature exclusion unit that creates a target feature obtained by excluding an image feature in the exclusion area from the image features extracted by the feature extraction unit; a feature collation unit that performs the collation processing between the target feature and the collation feature; and an output unit that outputs a result of the collation processing by the feature collation unit. Brief description of the drawings Fig. Figure 1 is a graphical representation of a hardware configuration of an object recognition device according to a first embodiment. Fig. Figure 2 is a graphical block representation depicting functions of the object recognition device according to the first embodiment. Fig. Figure 3 is a schematic view representing a user interface screen for determining an area in which an object is displayed. Fig. Figure 4 is a schematic view representing a user interface screen in a state where the area in which the object is displayed is determined. Fig. Figure 5 is a schematic view representing a user interface screen for determining an exclusion zone. Fig. Figure 6 is a schematic view showing a user interface screen in a state where an area is defined that shows what is to be excluded. Fig. Figure 7 is a schematic view that shows another example of the user interface screen in the state where the area is determined, showing what is to be excluded. Fig. Figure 8 is a schematic view that presents an example in which an edge feature is extracted as an image feature from an image obtained by capturing the object. Fig. Figure 9 is a schematic view that illustrates an example of a collation feature. Fig. Figure 10 is a schematic view that represents an example of an exclusion criterion. Fig. Figure 11 is a graphical block representation depicting functions of an object recognition device according to a second embodiment. Fig. Figure 12 is a schematic view that represents an example of an image containing an object, where the image is captured by an image capture unit. Fig. Figure 13 is a schematic view that presents an example in which image features are extracted by a feature extraction unit. Fig. Figure 14 is a schematic view that provides an example of image features obtained by partial exclusion using a feature exclusion unit. Fig. Figure 15 is a graphical block representation illustrating the functions of an object recognition device according to a third embodiment. Fig. Figure 16 is a schematic view that represents an example of an industrial machine. Description of embodiments

[0008] In the following, embodiments of the present disclosure are described with reference to the drawings. First embodiment

[0009] Fig. Figure 1 is a schematic graphical representation of a hardware configuration that constitutes a major part of an object detection device according to a first embodiment of the present disclosure. An object detection device 1 of the present disclosure can be mounted as a control device that controls an industrial machine, such as a robot, which detects the position and orientation of a workpiece and grasps the workpiece. Furthermore, the object detection device 1 of the present disclosure can be mounted on a computer, such as a personal computer provided together with the control device that controls the industrial machine, or a personal computer, a cell computer, a fog computer 6, a cloud server 7, or the like, which is connected to the control device via a wired or wireless network.In the present embodiment, an example is described in which the object recognition device 1 is mounted on the personal computer which is connected via the network to the control unit which controls the industrial machine.

[0010] A CPU 11 included in the object recognition device 1 according to the present embodiment is a processor that controls the object recognition device 1 as a whole. The CPU 11 reads a system program stored in a ROM 12 via a bus 22 and controls the entire object recognition device 1 according to the system program. A RAM 13 temporarily stores calculation data, display data, various types of externally entered data, and the like.

[0011] A non-volatile memory 14 is configured, for example, using a (not shown) battery-backed memory, a solid-state drive (SSD), and the like, and its memory state is maintained even when the power supply to the object detection device 1 is switched off. The non-volatile memory 14 stores a program and data read from an external device 72 via an interface 15, a program and data entered via an input device 71, programs and data acquired by industrial machines 3, and the like. The data stored in the non-volatile memory 14 can be developed in RAM 13 at the time it is executed and / or used. In addition, various system programs, such as a known analysis program, are pre-written in ROM 12.

[0012] Interface 15 is an interface for connecting the CPU 11 of the object recognition device 1 and the external device 72, such as a USB device. For example, a pre-stored control program, data relating to the operation of each of the industrial machines 3, and the like can be read from the external device 72. Furthermore, a control program, setting data, and the like, which are processed in the object recognition device 1, can be saved to an external storage device via the external device 72.

[0013] Interface 20 is an interface for connecting the CPU 11 of the object recognition device 1 and a wired or wireless network 5. The industrial machines 3, the fog computer 6, the cloud server 7 and the like are connected to the network 5 and exchange data with the object recognition device 1.

[0014] Data read from memory, data obtained as a result of program execution, and the like are output via an interface 17 and displayed on a display device 70. Furthermore, the input device 71, which includes a keyboard, a pointing device, and the like, transmits a command, data, and the like to the CPU 11 via an interface 18 based on operation by a worker.

[0015] Industrial machine 3 can be a machine that captures an image of an object, such as a workpiece. As in Fig. As shown in Figure 16, the industrial machine 3 can, for example, be a robot that includes an image sensor 4 and grasps an object 400 whose position and orientation have been specified based on an image captured by the image sensor 4. Furthermore, the industrial machine 3 can be a machine that captures an image of an object using the attached image sensor 4 and transmits the captured image to an appearance inspection device (not shown). The image of the object, captured by the image sensor 4 integrated into the industrial machine 3, is transmitted via the network 5 to the object recognition device 1.

[0016] Interface 21 is an interface for connecting the CPU 11 and a machine learning device 100. The machine learning device 100 has a processor 101 that controls the entire machine learning device 100, a ROM 102 that stores a system program and the like, a RAM 103 for temporary storage during any machine learning processing, and non-volatile memory 104 used to store a model and the like. The machine learning device 100 can observe, via interface 21, any piece of information that can be captured by the object recognition device 1.Furthermore, the object recognition device 1 captures a processing result output by the machine learning device 100 via interface 21, stores or displays the captured result and transmits the captured result to another device via network 5 or the like.

[0017] Fig. Figure 2 is a schematic graphical block diagram illustrating the functions of the object recognition device 1 according to the first embodiment of the present disclosure. The functions included in the object recognition device 1 according to the present embodiment are provided by the CPU 11 of the [unclear text]. Fig. 1 object recognition device 1 and the processor 101 of the machine learning device 100, which executes system programs and controls operations of the respective units of the object recognition device 1 and the machine learning device 100.

[0018] The object recognition device 1 of the present embodiment comprises an image acquisition unit 110, a range determination receiving unit 120, a feature extraction unit 125, a collation feature acquisition unit 130, an exclusion feature acquisition unit 140, and a learning unit 150. Furthermore, a collation feature storage unit 210, which is an area for storing a feature of an object, is pre-configured in the RAM 13 or in the non-volatile memory 14 of the object recognition device 1. Additionally, a classification parameter storage unit 220, which is an area for storing a parameter of a machine learning classifier that classifies image features into an object area and an exclusion area, is pre-configured in the RAM 103 or in the non-volatile memory 104 of the machine learning device 100.

[0019] The image acquisition unit 110 captures at least one image obtained by capturing an object. The object is, for example, a workpiece. The image acquisition unit 110 can capture multiple images obtained by capturing the object. Furthermore, the image acquisition unit 110 captures an image obtained by capturing at least one area that is to be excluded during collation processing of the object. The area to be excluded can, for example, include a table, a fixture, or the like on which the object is placed. The image acquisition unit 110 can capture multiple images obtained by capturing the area to be excluded. The image obtained by capturing the object and the image obtained by capturing the area to be excluded can be the same image. In addition, the images can be different.The image captured by the image acquisition unit 110 can be any image, provided that at least one operator viewing the image can identify the object and the area to be excluded within the image. The image acquisition unit 110 outputs the captured image to the area determination receiving unit 120 and the feature extraction unit 125.

[0020] The area determination receiver 120 receives a determination of the object area, which is the area within the total area of ​​the image captured by the image acquisition unit 110 in which the object is depicted. The area determination receiver 120 can provide a user interface for the operator to receive the object area determination. In this case, the area determination receiver 120 displays the image obtained by capturing the object and inputting it from the image acquisition unit 110 on the display device 70. The operator is then prompted to operate the input device 71 to determine the object area from the image obtained by capturing the object.

[0021] Fig. Figure 3 is a schematic view representing a user interface screen for determining an area in which the object is displayed. In the example of Fig. 3. An image 310, obtained by capturing the object, is displayed on a user interface screen 300 to determine the object's area. The image 310 then depicts a state in which an object 312 is placed on a base 314. When such a user interface screen 300 is displayed on the display device 70, the operator uses the pointing device of the input device 71 to define an area in which the object 312 is represented by means of a cursor 330. The operator can define the area in which the object 312 is represented by drawing with a drawing tool or the like.Furthermore, the area in which object 312 is represented in the image can be specified using a known algorithm such as graph sectioning, after an arbitrary position of object 312 has been determined or an area has been determined such that it is enclosed by a rectangle. Fig. Figure 4 is a schematic view representing a user interface screen in a state where the area in which object 312 is displayed is defined. Fig. 4 is a hatched area, the area in which object 312 is displayed, the area being determined by the operator. It should be noted that multiple object areas in which the object is displayed can be determined from a single image. Furthermore, the object area in which the object is displayed can be determined from multiple images.

[0022] Furthermore, the area determination receiver 120 receives a determination of the exclusion area, which is an area to be excluded from the entire area of ​​the image captured by the image acquisition unit 110. The area determination receiver 120 may provide a user interface for the operator to receive the exclusion area determination. In this case, the area determination receiver 120 displays on the display device 70 the image that is input by the image acquisition unit 110 and obtained by capturing the area to be excluded. The operator is then prompted to operate the input device 71 to determine the exclusion area from the image obtained by capturing the area to be excluded.

[0023] Fig. Figure 5 is a schematic view representing a user interface screen for determining the exclusion range. In the example of Fig. 5. An image 315, obtained by capturing the area to be excluded, is displayed on a user interface screen 305 to determine the area to be excluded. The image 315 then shows the state in which the object 312 is placed on the base 314. When such a user interface screen 305 is displayed on the display device 70, the operator uses the pointing device of the input device 71 to determine, with the cursor 330, an area that represents what is to be excluded during the collation processing of the object. The area to be excluded can be determined by painting with a drawing tool or the like.Furthermore, the area in the image where what is to be excluded is shown can be specified using a known algorithm such as graph cutting, after an arbitrary position of the area to be excluded has been determined, or after an area has been determined such that it is enclosed by a rectangle. Fig. Figure 6 is a schematic view representing a user interface screen in a state where the area is defined, showing what is to be excluded. Fig. 6 is a hatched area, the area to be excluded, where the area is determined by the operator. It should be noted that multiple exclusion areas can be defined, as shown in Fig. 7 is shown. In addition, the exclusion area can be determined from a plurality of images.

[0024] The feature extraction unit 125 extracts image features from the image captured by the image acquisition unit 110. An edge feature is one example of an image feature extracted by the feature extraction unit 125. A corner feature is another example of an image feature extracted by the feature extraction unit 125. Fig. Figure 8 is a schematic view that presents an example in which edge features are extracted as image features from the image obtained by capturing the in Fig. The feature extraction unit 125 outputs the image features extracted from the image obtained by capturing the object to the collation feature acquisition unit 130. Furthermore, the feature extraction unit 125 outputs image features extracted from the image obtained by capturing the area to be excluded to the exclusion feature acquisition unit 140. Finally, the feature extraction unit 125 outputs the extracted image features to the learning unit 150.

[0025] The collation feature acquisition unit 130 acquires as a collation feature an image feature that is contained in the object area determined by the area determination receiving unit 120, from the image features extracted by the feature extraction unit 125. Fig. Figure 9 is a schematic view that illustrates an example of the collation feature. The example of Fig. 9 is an example where the collation feature is derived from the in Fig. The 8 depicted image features are captured when the area in which the object is depicted is determined, as in Fig. Figure 4 illustrates this. If multiple object areas are defined, the collation feature acquisition unit 130 can acquire multiple collation features. The collation feature acquisition unit 130 stores the acquired collation features in the collation feature storage unit 210. Furthermore, the collation feature acquisition unit 130 outputs the acquired collation feature to the learning unit 150.

[0026] The exclusion feature detection unit 140 detects as an exclusion feature an image feature that is included in the exclusion area determined by the area determination receiving unit 120, from the image features extracted by the feature extraction unit 125. Fig. Figure 10 is a schematic view that illustrates an example of the exclusion criterion. The example of Fig. 10 is an example where the exclusion criterion is from the in Fig. The 8 depicted image features are captured when the area in which what is to be excluded is depicted is determined as in Fig. Figure 6 illustrates this. The exclusion feature capture unit 140 can capture multiple exclusion features in a case where multiple exclusion areas are determined. The exclusion feature capture unit 140 outputs the extracted exclusion feature to the learning unit 150.

[0027] The learning unit 150 performs machine learning based on the image features extracted by the feature extraction unit 125, the collation feature detected by the collation feature detection unit 130, and the exclusion feature detected by the exclusion feature detection unit 140. It then generates a classifier to classify areas where the collation feature and exclusion features appear among the image features. If multiple collation features and exclusion features are present, the classifier is further refined through training based on these multiple features.The classifier thus generated calculates at least one score indicating the probability that each pixel of the image representing the image feature is included in the collation feature's range, and another score indicating the probability that each pixel of the image representing the image feature is included in the exclusion feature's range. If the score indicating the probability of being included in the collation feature's range exceeds a predefined threshold, that pixel is classified within the collation feature's range. Furthermore, if the score indicating the probability of being included in the exclusion feature's range exceeds a predefined threshold, that pixel is classified within the exclusion feature's range.It should be noted that if both exceed the thresholds, the pixel can be classified into either of the areas with a higher score or into an area that does not belong to either the collation or exclusion feature. This learning can be implemented using a known segmentation technique with a neural network such as a fully convolutional network as a model. Alternatively, a segmentation technique using a known model such as a Vision Transformer or a DPT can also be employed.Furthermore, a simpler method can be used, for example, to employ a well-known classifier such as a Support Vector Machine (SVM) or a Gaussian Mixture Model (GMM), which classifies multiple feature values ​​useful for differentiation, such as pixel values, the intensity of an edge feature, and the density of nearby edge features. It should be noted that in cases where multiple exclusion features are present, a classifier can be generated that classifies an area containing each exclusion feature as an exclusion area of ​​a different category. The training unit 150 stores the parameter of the trained model in the classification parameter storage unit 220.

[0028] The object recognition device 1 according to the present embodiment, which has the configuration described above, generates a classifier that, based on an image obtained by capturing the object and an image obtained by capturing what is to be excluded, classifies an area in which the object is represented and an area in which what is to be excluded is represented. This classifier can be used to specify an area to be excluded from a collation processing target from the image obtained by capturing the object. If an image feature in the area to be excluded is excluded from the collation processing target, the accuracy of the collation processing can be improved to a certain extent.Furthermore, a load on collation processing can be prevented by limiting the area in which the collation processing is performed.

[0029] For example, fine features that appear due to the surface texture of a table, fixture, pallet, or similar surface on which the object is placed—such as dirt, scratches, light reflections, and the like—can be distinguished from a feature that appears on the outline of a workpiece. Furthermore, if these fine features—those appearing due to the surface texture of the table, fixture, or pallet, dirt, scratches, light reflections, and the like—are learned in advance as areas where exclusion features of different categories appear, it is possible to perform finer area classification and exclusion, and an improvement in object recognition accuracy can be expected. Second embodiment

[0030] The following describes an object recognition device according to a second embodiment. The object recognition device 1 according to the present embodiment has a similar hardware configuration to that of the object recognition device 1 according to the first embodiment.

[0031] Fig. Figure 11 is a schematic graphical block diagram illustrating the functions of the object recognition device 1 according to the second embodiment of the present disclosure. The functions included in the object recognition device 1 according to the present embodiment are provided by the CPU 11 of the Fig. 1 object recognition device 1 and the processor 101 of the machine learning device 100, which executes system programs and controls operations of the respective units of the object recognition device 1 and the machine learning device 100.

[0032] The object recognition device 1 of the present embodiment includes the image acquisition unit 110, the feature extraction unit 125, a region classification unit 155, a feature exclusion unit 160, a feature collation unit 170, and an output unit 180. Furthermore, the collation feature storage unit 210, which is a region in which a feature of an object is stored in advance, is prepared in the RAM 13 or in the non-volatile memory 14 of the object recognition device 1. Additionally, the classification parameter storage unit 220, which is a region in which a parameter of a machine learning processing of a classifier that classifies image features into an object region and an exclusion region, is prepared in advance in the RAM 103 or in the non-volatile memory 104 of the machine learning device 100, is prepared.

[0033] The image acquisition unit 110 according to the present embodiment captures an image obtained by recording the object. The image acquisition unit 110 outputs the captured image to the feature extraction unit 125.

[0034] The feature extraction unit 125 extracts image features from the image captured by the image acquisition unit 110. The feature extraction unit 125 outputs the image features extracted from the image obtained by capturing the object to the area classification unit 155 and the feature exclusion unit 160.

[0035] The area classification unit 155 is a classifier in which the parameter stored in the classification parameter storage unit 220 is set. This classifier is generated by the object recognition device 1 according to the first embodiment. The area classification unit 155 classifies the image features input by the feature extraction unit 125 for each pixel, thereby classifying at least one area to be excluded from the image features. This classification can be implemented, for example, by a known segmentation technique.The area classification unit 155 calculates, using the classifier stored in the classification parameter memory unit 220, at least one score indicating the probability that each pixel of the image representing the image feature is included in an area of ​​a collation feature, and one score indicating the probability that each pixel of the image representing the image feature is included in an area of ​​an exclusion feature. If the score indicating the probability of being included in the area of ​​the collation feature exceeds a predefined threshold, that pixel is classified in the area of ​​the collation feature. Furthermore, if the score indicating the probability of being included in the area of ​​the exclusion feature exceeds a predefined threshold, that pixel is classified in the area of ​​the exclusion feature.

[0036] Feature exclusion unit 160 creates a target feature obtained by excluding (deleting) an image feature from an area classified as an area to be excluded by area classification unit 155, from the image features extracted by feature extraction unit 125. This target feature is then the target of collation processing by feature collation unit 170. For example, feature exclusion unit 160 can exclude the image feature by filling the image feature appearing in the area to be excluded with a background color. Feature exclusion unit 160 outputs the created target feature to feature collation unit 170.

[0037] The feature collation unit 170 performs the collation processing of the collation feature stored in the collation feature storage unit 210 on the target feature created by the feature exclusion unit 160. Then, the object's position and orientation within the target feature are specified. Known techniques such as template matching, DP matching, and graph matching can be used for the collation processing of the collation feature. The feature collation unit 170 outputs a result of the collation processing to the output unit 180.

[0038] Output unit 180 outputs the object's position and orientation, specified as a result of collation processing by feature collation unit 170. Output unit 180 can, for example, display and output the object's position and orientation on display device 70. Furthermore, output unit 180 can transmit and output the position and orientation to a control device of industrial machine 3 or to a computer such as fog computer 6 or cloud server 7 via network 5. Additionally, the position and orientation can be recorded in a log area created in RAM 13 or in non-volatile memory 14.

[0039] Fig. Figure 12 is a schematic view that provides an example of the image containing the object captured by the image acquisition unit 110. Furthermore, Fig. 13 A schematic view representing an example in which image features are extracted from the image by the feature extraction unit 125. Fig. The image shown in Figure 12 is extracted. In a conventional object recognition device, collation processing is performed on image features extracted from the entire image, as shown in Figure 12. Fig. Figure 13 illustrates this. Therefore, it is necessary to perform a collation with each of the image features, which can lead to incorrect recognition or similar issues if similar image features are present. Furthermore, collation processing is time-consuming. Fig. Figure 14 is a schematic view that illustrates an example of image features obtained by partial exclusion using the feature exclusion unit 160. As in Fig. As shown in Figure 14, in the object recognition device 1 according to the present embodiment, some image features are excluded from the image features using the classifier, which has learned as an exclusion feature an image feature of an area that has been defined by an operator as the area to be excluded. Therefore, it is not necessary to perform a collation with an unnecessary image feature, and an improvement in recognition accuracy and a reduction in collation processing time are expected.

[0040] The object recognition device 1 according to the present embodiment, which has the configuration described above, can reduce the possibility of erroneous recognition and further improve recognition accuracy by excluding an unnecessary image feature from the image that is a recognition target. In addition, the time required for collation processing can be reduced. In the captured image, which includes the outer edge of the object, image features of a base, a fixture, and the like, as well as image features reflected due to the lighting conditions, are mixed with an image feature of the object itself. In cases where image features other than the object are included, the collation processing involves matching these various image features, thus increasing the processing load.Furthermore, if an image feature similar to the object appears in the edge region, the accuracy of the collation processing may decrease. The object recognition device 1 according to the present embodiment can classify the area to be excluded from the image features using a classifier created based on the operator's specifications, and exclude the image feature appearing in that area from the target of the collation processing. In addition, unnecessary image features generated by various factors can be precisely excluded by defining a plurality of areas with exclusion feature regions. Third embodiment

[0041] The following describes an object recognition device according to a third embodiment. The object recognition device 1 according to the present embodiment has a similar hardware configuration to that of the object recognition device 1 according to the first embodiment.

[0042] Fig. Figure 15 is a schematic graphical block diagram illustrating the functions of the object recognition device 1 according to the third embodiment of the present disclosure. The functions included in the object recognition device 1 according to the present embodiment are provided by the CPU 11 of the [unclear text]. Fig. 1 object recognition device 1 and the processor 101 of the machine learning device 100, which executes system programs and controls operations of the respective units of the object recognition device 1 and the machine learning device 100.

[0043] The object recognition device 1 of the present embodiment further includes a threshold detection unit 190 in addition to the image detection unit 110, the feature extraction unit 125, the area classification unit 155, a feature exclusion unit 160, the feature collation unit 170 and the output unit 180. In addition, the collation feature storage unit 210, which is an area in which a feature of an object is stored in advance, is prepared in the RAM 13 or in the non-volatile memory 14 of the object recognition device 1.Furthermore, in the RAM 103 or in the non-volatile memory 104 of the device 100 for machine learning, the classification parameter storage unit 220 is prepared, which is an area in which a parameter of a machine learning processing of a classifier, which classifies image features into an object area and an exclusion area, is stored in advance.

[0044] The image acquisition unit 110, the feature extraction unit 125, the feature exclusion unit 160, the feature collation unit 170 and the output unit 180 according to the present embodiment have similar functions to those of the image acquisition unit 110, the feature extraction unit 125, the area classification unit 155, the feature collation unit 170 and the output unit 180 according to the second embodiment.

[0045] The threshold detection unit 190 detects thresholds used by the range classification unit 155. Examples of thresholds used by the range classification unit 155 include a target feature index threshold, indicating a possibility of being a target feature, and an exclusion feature index threshold, indicating a possibility of being an exclusion feature. The threshold detection unit 190 can detect these thresholds, for example, via a predefined user interface displayed on the display device 70. Alternatively, the thresholds can also be detected by a host computer, such as the fog computer 6 or the cloud server 7.

[0046] The area classification unit 155 according to the present embodiment classifies the image features input by the feature extraction unit 125 for each pixel, thereby classifying at least one area to be excluded from the image features. The area classification unit 155 classifies each area using the threshold value, which serves as the target feature index, and the threshold value, which serves as the exclusion feature index, both of which are detected by the threshold detection unit 190.

[0047] When excluding an unnecessary image feature from an image that is a recognition target, the object recognition device 1 according to the present embodiment, which has the configuration described above, can adjust the thresholds that enable a suitable classification of an exclusion area, for example by displaying a user interface to an operator and prompting the operator to enter the threshold value that serves as the target feature index and the threshold value that serves as the exclusion feature index and to confirm an area excluded in this way. Other embodiment

[0048] In the first embodiment described above, the object recognition device 1, which includes the configuration for training the classifier, has been described. Furthermore, the second embodiment describes an example in which the object in the image is recognized using the classifier trained by the object recognition device 1 of the first embodiment. However, the functions of the object recognition device 1 of the first embodiment and the object recognition device 1 of the second embodiment can be configured in a single object recognition device 1. With this configuration, the training of the classifier and the collation processing using the classifier can be performed by a single object recognition device 1.

[0049] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. These embodiments may be supplemented, replaced, modified, partially deleted, and the like in various ways without deviating from the essence of the invention or from the idea and essence of the present disclosure as described in the claims and their equivalents. For example, the sequence of operations and the sequence of processes are presented as examples in the embodiments described above and are not limited to them. The same applies if numerical values ​​or numerical expressions are used to describe the embodiments described above.

[0050] With regard to the embodiments described above, further remarks are disclosed below. Supplementary Note 1

[0051] An object recognition device (1), as described in an embodiment of the present disclosure, comprises an image acquisition unit (110) that acquires a first image obtained by picking up an object; a feature extraction unit (125) that extracts image features from the first image; and an area classification unit (155) that classifies an area in which an exclusion feature appears among the image features extracted by the feature extraction unit (125) as an exclusion area by means of a classifier that classifies an area in which a collation feature appears and the area in which the exclusion feature appears based on the collation feature, which indicates a feature of an image area that is to be a target of collation processing, and the exclusion feature, which is a feature of an image area that is to be excluded from the target of collation processing.a feature exclusion unit (160) that creates a target feature obtained by excluding an image feature in the exclusion area from the image features extracted by the feature extraction unit (125); a feature collation unit (170) that performs the collation processing between the target feature and the collation feature; and an output unit (180) that outputs a result of the collation processing by the feature collation unit (170). Supplementary Note 2

[0052] The object recognition device (1), as described in a further embodiment of the present disclosure, wherein the image acquisition unit (110) further acquires a second image on which an area to be excluded from collation processing is shown, and the feature extraction unit (125) extracts image features from both the first image and the second image, wherein the object recognition device (1) further comprises an area determination receiving unit (120) which receives a determination of at least one object area in the first image, which is an area in which the object is shown, and at least one exclusion area in the second image, which is an area to be excluded from a target of collation processing;a collation feature acquisition unit (130) that acquires at least one collation feature, which is an image feature of the object, based on the image features extracted from the first image by the feature extraction unit (125) and the at least one object area received by the area determination receiver unit (120); an exclusion feature acquisition unit (140) that acquires at least one exclusion feature, which is an image feature of the exclusion area, based on the image features extracted from the second image by the feature extraction unit (125) and the at least one exclusion area received by the area determination receiver unit (120);and a learning unit (150) that generates a classifier that classifies the object area and the exclusion area among the image features based on the image features extracted by the feature extraction unit (125), the at least one collation feature and the at least one exclusion feature. Supplementary Note 3

[0053] The object recognition device (1) as described in a further embodiment of the present disclosure, wherein the area determination receiving unit (120) receives a plurality of the exclusion areas. Supplementary note 4

[0054] The object recognition device (1), as described in a further embodiment of the present disclosure, wherein the learning unit (150) generates a classifier that classifies the plurality of exclusion areas into different categories. Supplementary note 5

[0055] The object recognition device (1) as described in a further embodiment of the present disclosure, wherein the image acquisition unit (110) acquires a plurality of the second images. Supplementary Note 6

[0056] The object recognition device (1), as described in a further embodiment of the present disclosure, wherein the image acquisition unit (110) acquires a plurality of the first images, the feature extraction unit (125) extracts image features from the plurality of first images, and the area determination receiving unit (120) receives a determination of object areas in the plurality of first images. Supplementary note 7

[0057] The object recognition device (1), as described in a further embodiment of the present disclosure, which further comprises a threshold detection unit (190) which detects a threshold for a target feature index indicating a possibility of being the target feature and / or a threshold for an exclusion feature index indicating a possibility of being the exclusion feature, wherein the area classification unit (155) classifies the area in which the exclusion feature appears as an exclusion area based on a threshold received by the threshold detection unit (190).

[0058] A computer-readable recording medium, as described in one embodiment of the present disclosure, which stores a program for causing a computer to operate as: an image acquisition unit (110) that acquires a first image obtained by capturing an object; a feature extraction unit (125) that extracts image features from the first image; an area classification unit (155) that classifies as an exclusion area an area in which an exclusion feature appears among the image features extracted by the feature extraction unit (125), using a classifier that classifies an area in which a collation feature appears and the area in which the exclusion feature appears, based on the collation feature, which specifies a feature of an image area to be the target of collation processing, and the exclusion feature, which is a feature of an image area to be excluded from the target of collation processing;a feature exclusion unit (160) that creates a target feature obtained by excluding an image feature in the exclusion area from the image features extracted by the feature extraction unit (125); a feature collation unit (170) that performs the collation processing between the target feature and the collation feature; and an output unit (180) that outputs a result of the collation processing by the feature collation unit. List of reference symbols 1 object recognition device 3 industrial machines 4 image sensors 5 Network 6 Fog Computer 7 cloud servers 11 CPU 12 ROM 13 RAM 14 non-volatile memory 15 Interface 17, 18, 20, 21 Interface 22 Bus 70 Display device 71 Input device 72 external device 100 Device for machine learning 101 processor 102 ROM 103 RAM 104 non-volatile memory 110 image capture units 120 Area Determination Receiver Unit 125 feature extraction unit 130 Collation feature recording unit 140 Exclusion Feature Recording Unit 150 learning units 155 Area classification unit 160 feature exclusion unit 170 feature collation unit 180 output units 210 collation feature storage unit 220 classification parameter storage unit QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2002-319022 A

[0003]

Claims

[1] Object recognition device comprising: an image capture unit that captures an initial image obtained by recording an object; a feature extraction unit that extracts image features from the first image; a range classification unit that classifies as an exclusion range an area in which an exclusion feature appears among the image features extracted by the feature extraction unit, using a classifier that classifies an area in which a collation feature appears and the area in which the exclusion feature appears, based on the collation feature that specifies a feature of an image area that is to be a target of collation processing and the exclusion feature that is a feature of an image area that is to be excluded from the target of collation processing; a feature exclusion unit that creates a target feature obtained by excluding an image feature in the exclusion area from the image features extracted by the feature extraction unit; a feature collation unit that performs collation processing between the target feature and the collation feature; and an output unit that outputs a result of the collation processing by the feature collation unit. [2] Object recognition device according to claim 1, wherein The image acquisition unit also captures a second image showing an area to be excluded from collation processing, and The feature extraction unit extracts image features from both the first and second images. the object recognition device further comprises: a range determination receiver unit that receives a determination of at least one object range in the first image, which is a range in which the object is represented, and at least one exclusion range in the second image, which is a range to be excluded from a collation processing target; a collation feature acquisition unit that acquires at least one collation feature, which is an image feature of the object, based on the image features extracted from the first image by the feature extraction unit and the at least one object area received by the area determination receiving unit; an exclusion feature detection unit that detects at least one exclusion feature, which is an image feature of the exclusion area, based on the image features extracted from the second image by the feature extraction unit and the at least one exclusion area received by the area determination receiving unit; and a learning unit that generates a classifier that classifies the object area and the exclusion area among the image features based on the image features extracted by the feature extraction unit, the at least one collation feature, and the at least one exclusion feature. [3] Object recognition device according to claim 2, wherein the area determination receiving unit receives a plurality of the exclusion areas. [4] Object recognition device according to claim 3, wherein the learning unit generates a classifier that classifies the plurality of exclusion areas into different categories. [5] Object recognition device according to claim 2, wherein the image acquisition unit acquires a plurality of the second images. [6] Object recognition device according to claim 2, wherein the image capture unit captured a majority of the first images, The feature extraction unit extracts image features from the majority of initial images, and The area determination receiver receives a determination of object areas in the majority of first images. [7] Object recognition device according to claim 1, further comprising a threshold detection unit which detects a threshold for a target feature index indicating a possibility of being the target feature and / or a threshold for an exclusion feature index indicating a possibility of being the exclusion feature, wherein the area classification unit classifies the area in which the exclusion feature appears as an exclusion area based on a threshold received by the threshold detection unit. [8] Computer-readable recording medium that stores a program to cause a computer to operate as: an image capture unit that captures an initial image obtained by recording an object; a feature extraction unit that extracts image features from the first image; a range classification unit that classifies as an exclusion range an area in which an exclusion feature appears among the image features extracted by the feature extraction unit, using a classifier that classifies an area in which a collation feature appears and the area in which the exclusion feature appears, based on the collation feature that specifies a feature of an image area that is to be a target of collation processing and the exclusion feature that is a feature of an image area that is to be excluded from the target of collation processing; a feature exclusion unit that creates a target feature obtained by excluding an image feature in the exclusion area from the image features extracted by the feature extraction unit; a feature collation unit that performs collation processing between the target feature and the collation feature; and an output unit that outputs a result of the collation processing by the feature collation unit.

Images