Object detection device and computer-readable recording medium

Abstract

An object detection device according to the present disclosure comprises: an image acquisition unit that acquires a first image obtained by capturing an image of an object; a feature extraction unit that extracts image features from the first image; a region classification unit in which comparison features indicating features of an image region to be subjected to comparison processing and an exclusion feature which is a feature of an image region to be excluded from the comparison processing are used as a basis by a classifier for classifying regions in which the comparison features among the image features appear and regions in which the exclusion feature appears to classify a region in which the exclusion feature among the image features extracted by the feature extraction unit appears as an exclusion region; a feature exclusion unit that creates target features obtained by excluding image features within the exclusion region from the image features extracted by the feature extraction unit; a feature comparison unit that performs comparison processing between the target features and the comparison features; and an output unit that outputs the result of the comparison processing by the feature comparison unit.

Description

Object detection device and computer-readable recording medium

[0001] The present disclosure relates to an object detection device and a computer-readable recording medium.

[0002] When a robot works on an object such as a workpiece whose position and orientation are unknown, the position and orientation of the object may be identified using an image (see, for example, Patent Document 1). For example, image features representing specific locations of the object are extracted from an image of an object whose position and orientation are known, and the extracted image features are registered together with the position and orientation. For an object whose position and orientation are unknown, image features representing specific locations of the object are similarly extracted from the captured image, and the position and orientation of the object are identified from the amount of change in position and orientation on the image obtained by comparing the image features with the registered image features (model features) and the registered position and orientation. Image features used for comparison include edge features representing locations with large brightness differences and corner features representing the intersection of two or more edges.

[0003] Japanese Patent Application Laid-Open No. 2002-319022

[0004] When the image features (model features) used for matching are not prominent in an image, for example, a threshold used to determine whether to extract the desired image features may be set low, resulting in the extraction of many image features other than the desired location in the image. This can result in areas where the model features closely match the wrong location, resulting in the erroneous detection of the target object. Another problem is that the processing time required for matching image features increases. Therefore, a technology that improves the accuracy of detecting targets in images is desired.

[0005] The object detection device according to the present disclosure solves the above-mentioned problems by learning to classify image features extracted by a feature extraction process into matching features that indicate characteristics of the object and exclusion features that indicate unnecessary features. When detecting an object, the features of the acquired image are classified into matching features and exclusion features, and the image features classified as exclusion features are excluded before the matching process.

[0006] One aspect of the present disclosure is an object detection device including: an image acquisition unit that acquires a first image of an object; a feature extraction unit that extracts image features from the first image; a region classification unit that classifies, as an exclusion region, a region among the image features extracted by the feature extraction unit in which an exclusion feature appears, using a classifier that classifies regions in the image features in which a matching feature appears and a region in which an exclusion feature appears based on matching features that indicate characteristics of an image region that should be the subject of the matching process and exclusion features that are characteristics of an image region that should be excluded from the subject of the matching process; a feature exclusion unit that creates a target feature from the image features extracted by the feature extraction unit, excluding the image features in the exclusion region; a feature matching unit that performs a matching process between the target feature and the matching feature; and an output unit that outputs a result of the matching process by the feature matching unit.

[0007] 1 is a hardware configuration diagram of an object detection device according to a first embodiment; FIG. 2 is a block diagram showing functions of the object detection device according to the first embodiment; FIG. 3 is a schematic diagram illustrating a user interface screen for specifying an area in which an object is captured; FIG. 4 is a schematic diagram illustrating a user interface screen in a state where an area in which an object is captured has been specified; FIG. 5 is a schematic diagram illustrating a user interface screen for specifying an exclusion area; FIG. 6 is a schematic diagram illustrating a user interface screen in a state where an area in which an object to be excluded is captured has been specified; FIG. 7 is a schematic diagram showing another example of a user interface screen in a state where an area in which an object to be excluded is captured has been specified; FIG. 8 is a schematic diagram showing an example of edge features extracted as image features from an image capturing an object; FIG. 9 is a schematic diagram showing an example of matching features; FIG. 10 is a block diagram showing functions of an object detection device according to a third embodiment; FIG. 11 is a schematic diagram showing an example of industrial machinery;

[0008] Embodiments of the present disclosure will be described below with reference to the drawings. [First Embodiment] Fig. 1 is a schematic hardware configuration diagram showing the main components of an object detection device according to a first embodiment of the present disclosure. The object detection device 1 of the present disclosure can be implemented as a control device that controls industrial machinery, such as a robot that detects the position and orientation of a workpiece and grips it. The object detection device 1 of the present disclosure can also be implemented on a personal computer (PC) attached to the control device that controls the industrial machinery, or on a computer such as a personal computer, cell computer, fog computer 6, or cloud server 7 connected to the control device via a wired or wireless network. In this embodiment, an example is shown in which the object detection device 1 is implemented on a PC connected to the control device that controls the industrial machinery via a network.

[0009] The CPU 11 provided in the object detection device 1 according to this embodiment is a processor that controls the entire object detection device 1. The CPU 11 reads a system program stored in the ROM 12 via the bus 22, and controls the entire object detection device 1 in accordance with the system program. The RAM 13 temporarily stores temporary calculation data, display data, and various data input from outside.

[0010] The non-volatile memory 14 is configured, for example, by a battery-backed memory (not shown) or an SSD (Solid State Drive), and retains its stored state even when the power to the object detection device 1 is turned off. The non-volatile memory 14 stores programs and data read from an external device 72 via the interface 15, programs and data input via the input device 71, programs and data acquired from the industrial machine 3, and the like. The data stored in the non-volatile memory 14 may be expanded into the RAM 13 during execution / use. In addition, various system programs such as known analysis programs are written in the ROM 12 in advance.

[0011] The interface 15 is an interface for connecting the CPU 11 of the object detection device 1 to an external device 72 such as a USB device. For example, pre-stored control programs and data related to the operation of each industrial machine 3 can be read from the external device 72. In addition, control programs and setting data edited within the object detection device 1 can be stored in external storage means via the external device 72.

[0012] The interface 20 is an interface for connecting the CPU 11 of the object detection device 1 to a wired or wireless network 5. The network 5 is connected to the industrial machines 3, the fog computer 6, the cloud server 7, etc., and exchanges data with the object detection device 1.

[0013] The display device 70 displays various data loaded into the memory, data obtained as a result of executing programs, etc., output via the interface 17. An input device 71, which is comprised of a keyboard, pointing device, etc., passes instructions, data, etc., based on operations by an operator to the CPU 11 via the interface 18.

[0014] The industrial machine 3 may be a machine that captures an image of an object such as a workpiece. For example, as illustrated in Fig. 16 , the industrial machine 3 may be a robot that is equipped with an image sensor 4 and grasps an object 400 whose position and orientation have been determined based on an image captured by the image sensor 4. The industrial machine 3 may also be a machine that captures an image of an object using the attached image sensor 4 and transmits the captured image to a visual inspection device (not shown). The image including the object captured by the image sensor 4 provided in the industrial machine 3 is transmitted to the object detection device 1 via the network 5.

[0015] The interface 21 is an interface for connecting the CPU 11 and the machine learning device 100. The machine learning device 100 includes a processor 101 that controls the entire machine learning device 100, a ROM 102 that stores system programs and the like, a RAM 103 that provides temporary storage for each process related to machine learning, and a non-volatile memory 104 that stores models and the like. The machine learning device 100 can observe each piece of information that can be acquired by the object detection device 1 via the interface 21. The object detection device 1 also acquires processing results output from the machine learning device 100 via the interface 21, and stores or displays the acquired results, or transmits them to other devices via the network 5 or the like.

[0016] 2 is a schematic block diagram illustrating functions of the object detection device 1 according to the first embodiment of the present disclosure. Each function of the object detection device 1 according to this embodiment is realized by the CPU 11 of the object detection device 1 and the processor 101 of the machine learning device 100 shown in FIG. 1 executing a system program and controlling the operation of each part of the object detection device 1 and the machine learning device 100.

[0017] The object detection device 1 of this embodiment includes an image acquisition unit 110, an area designation receiving unit 120, a feature extraction unit 125, a matching feature acquisition unit 130, an exclusion feature acquisition unit 140, and a learning unit 150. A matching feature storage unit 210, which is an area for storing object features, is prepared in advance on the RAM 13 to the nonvolatile memory 14 of the object detection device 1. Furthermore, a classification parameter storage unit 220, which is an area for storing machine learning processing parameters of a classifier that classifies image features into object areas and exclusion areas, is prepared in advance on the RAM 103 to the nonvolatile memory 104 of the machine learning device 100.

[0018] The image acquisition unit 110 acquires an image of at least an object. The object is, for example, a workpiece. The image acquisition unit 110 may acquire multiple images of the object. Furthermore, the image acquisition unit 110 acquires an image of at least an area to be excluded in the object matching process. The area to be excluded may include, for example, a table or a jig on which the object is placed. The image acquisition unit 110 may acquire multiple images of the area to be excluded. The image of the object and the image of the area to be excluded may be the same image, or may be different images. The image acquired by the image acquisition unit 110 only needs to allow an operator viewing the image to identify the object and the area to be excluded in the image. The image acquisition unit 110 outputs the acquired image to the area designation acceptance unit 120 and the feature extraction unit 125.

[0019] The area designation receiving unit 120 receives designation of an object area, which is an area in which an object is captured, from the entire area of ​​the image acquired by the image acquisition unit 110. The area designation receiving unit 120 may provide the operator with a user interface for receiving the designation of the object area. In this case, the area designation receiving unit 120 displays on the display device 70 the image of the object input from the image acquisition unit 110. The operator is then prompted to operate the input device 71 to designate an object area from the image of the object.

[0020] FIG. 3 is a schematic diagram illustrating a user interface screen for specifying an area in which an object is captured. In the example of FIG. 3 , a user interface screen 300 for specifying an object area displays an image 310 of an object captured. An object 312 placed on a pedestal 314 is shown on the image 310. When such a user interface screen 300 is displayed on the display device 70, the operator operates the pointing device of the input device 71 to specify the area in which the object 312 is captured with a cursor 330. At this time, the operator may specify the area in which the object 312 is captured by painting with a paint tool or the like. Alternatively, the operator may specify any position of the object 312 or specify an area by enclosing it in a rectangle, and then use a known algorithm such as graph cut to identify the area in which the object 312 is captured within the image. FIG. 4 is a schematic diagram illustrating a user interface screen in which the area in which the object 312 is captured has been specified. In FIG. 4 , the shaded area indicates the area in which the object 312 is captured and specified by the operator. It is also possible to designate a plurality of object regions in which an object is captured from one image, or to designate a plurality of object regions in which an object is captured from multiple images.

[0021] Furthermore, the region designation receiving unit 120 receives designation of an exclusion region, which is a region that should be excluded from the entire region of the image acquired by the image acquisition unit 110. The region designation receiving unit 120 may provide the operator with a user interface that receives the designation of the exclusion region. In this case, the region designation receiving unit 120 displays, on the display device 70, an image of the region that should be excluded that is input from the image acquisition unit 110. The operator is then prompted to operate the input device 71 to designate the exclusion region from the image of the region that should be excluded.

[0022] FIG. 5 is a schematic diagram illustrating a user interface screen for specifying an exclusion area. In the example of FIG. 5, a user interface screen 305 for specifying an exclusion area displays an image 315 of an area to be excluded. An object 312 is shown placed on a pedestal 314 on the image 315. When such a user interface screen 305 is displayed on the display device 70, the operator operates the pointing device of the input device 71 to use a cursor 330 to specify an area containing an object to be excluded in the object matching process. At this time, the area to be excluded may be specified by painting it with a paint tool or the like. Alternatively, a position of the area to be excluded may be specified, or a range may be specified by enclosing it in a rectangle, and then a known algorithm such as graph cut may be used to identify the area in the image containing the object to be excluded. FIG. 6 is a schematic diagram illustrating a user interface screen in which an area containing an object to be excluded has been specified. In FIG. 6, the shaded area is the area to be excluded specified by the operator. Note that, as illustrated in FIG. 7, multiple exclusion areas may be specified. Furthermore, exclusion areas may be specified from multiple images.

[0023] The feature extraction unit 125 extracts image features from the image acquired by the image acquisition unit 110. The image features extracted by the feature extraction unit 125 are, for example, edge features. Furthermore, the image features extracted by the feature extraction unit 125 are, for example, corner features. FIG. 8 is a schematic diagram showing an example in which edge features are extracted as image features from an image of the object illustrated in FIG. 3 . The feature extraction unit 125 outputs the image features extracted from the image of the object to the matching feature acquisition unit 130. Furthermore, the feature extraction unit 125 outputs the image features extracted from the image of the area to be excluded to the exclusion feature acquisition unit 140. Furthermore, the feature extraction unit 125 outputs the extracted image features to the learning unit 150.

[0024] The matching feature acquisition unit 130 acquires, as matching features, image features included in the object region specified by the region designation receiving unit 120 from the image features extracted by the feature extraction unit 125. FIG. 9 is a schematic diagram showing an example of matching features. The example of FIG. 9 is an example in which a matching feature is acquired from the image features exemplified in FIG. 8 when a region containing an object is specified as exemplified in FIG. 4. The matching feature acquisition unit 130 may acquire multiple matching features when multiple object regions are specified. The matching feature acquisition unit 130 stores the acquired matching features in the matching feature storage unit 210. Furthermore, the matching feature acquisition unit 130 outputs the acquired matching features to the learning unit 150.

[0025] The exclusion feature acquisition unit 140 acquires, from the image features extracted by the feature extraction unit 125, image features included in the exclusion area specified by the area designation receiving unit 120 as exclusion features. Fig. 10 is a schematic diagram showing an example of exclusion features. The example of Fig. 10 is an example in which exclusion features are acquired from the image features exemplified in Fig. 8 when an area containing an object to be excluded is specified as exemplified in Fig. 6. The exclusion feature acquisition unit 140 may acquire multiple exclusion features when multiple exclusion areas are specified. The exclusion feature acquisition unit 140 outputs the extracted exclusion features to the learning unit 150.

[0026] [Correction Based on Rule 91, 12.09.2024] The learning unit 150 performs machine learning processing based on the image features extracted by the feature extraction unit 125, the matching features acquired by the matching feature acquisition unit 130, and the excluded features acquired by the excluded feature acquisition unit 140 to generate a classifier for classifying regions in which matching features and excluded features appear among the image features. When there are multiple target features and excluded features, the classifier is created by performing learning based on these multiple target features and excluded features. The generated classifier calculates at least one of a score indicating the likelihood that each pixel in the image exhibiting the image features is included in the matching feature region and a score indicating the likelihood that the pixel is included in the excluded feature region. If the score indicating the likelihood that the pixel is included in the matching feature region exceeds a predetermined threshold, the pixel is classified as a matching feature region. Furthermore, if the score indicating the likelihood that the pixel is included in the excluded feature region exceeds a predetermined threshold, the pixel is classified as an excluded feature region. Note that if both scores exceed the threshold, the pixel may be classified into the region with the higher score, or into a region that does not belong to either the matching feature region or the excluded feature region. This learning can be achieved by using a known segmentation technique modeled on a neural network, such as a fully convolutional network. Also, segmentation techniques using known models, such as Vision Transformer and DPT, can be used. Furthermore, as a simpler method, known classifiers, such as a Support Vector Machine (SVM) or a Gaussian Mixture Model (GMM), can be used to classify multiple feature quantities useful for discrimination, such as pixel values, edge feature strength, and density of nearby edge features. When there are multiple exclusion features, a classifier may be generated that classifies regions in which each exclusion feature appears as an exclusion region of a different category. The learning unit 150 stores the parameters of the learned model in the classification parameter storage unit 220.

[0027] The object detection device 1 according to the present embodiment, which has the above-described configuration, generates a classifier that classifies areas containing the object and areas containing the object to be excluded based on an image of the object and an image of an object to be excluded. This classifier can be used to identify areas in the image of the object to be excluded from the matching process. By excluding image features in the areas to be excluded from the matching process, it is possible to improve the accuracy of the matching process to a certain extent. Furthermore, by limiting the range of the matching process, it is possible to reduce the load of the matching process.

[0028] For example, it is possible to distinguish fine features that appear in the surface structure, dirt, scratches, light reflection, etc. of the table, jig, pallet, etc. on which the object is placed from features that appear in the contour line of the workpiece. By further learning in advance the fine features that appear in the surface structure, dirt, scratches, light reflection, etc. of the table, jig, pallet, etc. as areas where different categories of exclusion features appear, it becomes possible to perform more detailed region classification and region exclusion, which is expected to improve the accuracy of object detection.

[0029] Second Embodiment An object detection device according to a second embodiment will be described below. The object detection device 1 according to this embodiment has the same hardware configuration as the object detection device 1 according to the first embodiment.

[0030] 11 is a schematic block diagram illustrating functions provided in the object detection device 1 according to the second embodiment of the present disclosure. Each function provided in the object detection device 1 according to this embodiment is realized by the CPU 11 of the object detection device 1 and the processor 101 of the machine learning device 100 shown in FIG. 1 executing a system program and controlling the operation of each part of the object detection device 1 and the machine learning device 100.

[0031] The object detection device 1 of this embodiment includes an image acquisition unit 110, a feature extraction unit 125, an area classification unit 155, a feature exclusion unit 160, a feature matching unit 170, and an output unit 180. A matching feature storage unit 210, which is an area in which object features are pre-stored, is provided on the RAM 13 to the non-volatile memory 14 of the object detection device 1. Furthermore, a classification parameter storage unit 220, which is an area in which machine learning processing parameters of a classifier that classifies image features into object areas and exclusion areas, is pre-stored, is provided on the RAM 103 to the non-volatile memory 104 of the machine learning device 100.

[0032] The image acquisition unit 110 according to this embodiment acquires an image of an object. The image acquisition unit 110 acquires an image of an object. The image acquisition unit 110 outputs the acquired image to the feature extraction unit 125.

[0033] The feature extraction unit 125 extracts image features from the image acquired by the image acquisition unit 110. The feature extraction unit 125 outputs the image features extracted from the image of the object to the region classification unit 155 and the feature exclusion unit 160.

[0034] The region classification unit 155 is a classifier to which parameters stored in the classification parameter storage unit 220 are set. This classifier is generated by the object detection device 1 according to the first embodiment. The region classification unit 155 classifies the image features input from the feature extraction unit 125 on a pixel-by-pixel basis to classify at least the region of the image features to be excluded. This classification can be achieved, for example, by known segmentation techniques. The region classification unit 155 uses the classifier stored in the classification parameter storage unit 220 to calculate at least one of a score indicating the likelihood that each pixel of the image exhibiting the image features is included in the region of the matching feature and a score indicating the likelihood that the pixel is included in the region of the exclusion feature. If the score indicating the likelihood that the pixel is included in the region of the matching feature exceeds a predetermined threshold, the region classification unit 155 classifies the pixel into the region of the matching feature. If the score indicating the likelihood that the pixel is included in the region of the exclusion feature exceeds a predetermined threshold, the region classification unit 155 classifies the pixel into the region of the exclusion feature.

[0035] The feature exclusion unit 160 creates target features by excluding (deleting) image features of regions that the region classification unit 155 has classified as regions to be excluded from the image features extracted by the feature extraction unit 125. These target features are the targets of matching processing by the feature matching unit 170. The feature exclusion unit 160 may exclude image features, for example, by filling in image features that appear in regions to be excluded with a background color. The feature exclusion unit 160 outputs the created target features to the feature matching unit 170.

[0036] The feature matching unit 170 performs a matching process on the target features created by the feature exclusion unit 160 using the matching features stored in the matching feature storage unit 210. Then, the feature matching unit 170 identifies the position and orientation of the target object in the target features. The matching process on the matching features can use known techniques such as template matching, DP matching, and graph matching. The feature matching unit 170 outputs the results of the matching process to the output unit 180.

[0037] The output unit 180 outputs the position and orientation of the object identified as a result of the matching process by the feature matching unit 170. The output unit 180 may, for example, display the position and orientation of the object on the display device 70. Alternatively, the output unit 180 may transmit the position and orientation of the object to a control device of the industrial machine 3, or a computer such as the fog computer 6 or the cloud server 7 via the network 5. Furthermore, the output unit 180 may record the position and orientation of the object in a log area provided in the RAM 13 or the non-volatile memory 14.

[0038] FIG. 12 is a schematic diagram illustrating an example of an image including an object acquired by the image acquisition unit 110. FIG. 13 is a schematic diagram illustrating an example of image features extracted by the feature extraction unit 125 from the image illustrated in FIG. 12. In conventional object detection devices, as illustrated in FIG. 13, a matching process was performed on image features extracted from the entire image. Therefore, matching between each image feature was necessary, which could result in erroneous detection when similar image features were included. Furthermore, the matching process took a long time. FIG. 14 is a schematic diagram illustrating an example of image features partially excluded by the feature exclusion unit 160. As illustrated in FIG. 14, the object detection device 1 according to this embodiment uses a classifier trained to exclude image features from areas designated by the operator as exclusion features, thereby excluding some image features from the image features. This eliminates the need for matching between unnecessary image features, which is expected to improve detection accuracy and shorten the matching process.

[0039] The object detection device 1 according to this embodiment, configured as described above, can reduce the possibility of false detection and improve detection accuracy by eliminating unnecessary image features from the image to be detected. It can also shorten the time required for the matching process. Images captured including the area surrounding the object contain not only image features of the object itself, but also image features of the base, tool, and other components, as well as image features captured due to lighting conditions. When image features other than the object are included, the matching process involves matching various image features, increasing the processing load. Furthermore, the accuracy of the matching process may decrease if image features similar to the object appear in the vicinity. The object detection device 1 according to this embodiment uses a classifier created based on operator specifications to classify areas to be excluded from image features and exclude image features appearing in those areas from the matching process. Furthermore, by specifying multiple excluded feature areas, unnecessary image features arising from various factors can be accurately excluded.

[0040] Third Embodiment An object detection device according to a third embodiment will be described below. The object detection device 1 according to this embodiment has the same hardware configuration as the object detection device 1 according to the first embodiment.

[0041] 15 is a schematic block diagram illustrating functions provided in the object detection device 1 according to the third embodiment of the present disclosure. Each function provided in the object detection device 1 according to this embodiment is realized by the CPU 11 of the object detection device 1 and the processor 101 of the machine learning device 100 shown in FIG. 1 executing a system program and controlling the operation of each part of the object detection device 1 and the machine learning device 100.

[0042] The object detection device 1 of this embodiment includes an image acquisition unit 110, a feature extraction unit 125, an area classification unit 155, a feature exclusion unit 160, a feature matching unit 170, and an output unit 180, as well as a threshold acquisition unit 190. A matching feature storage unit 210, which is an area in which object features are pre-stored, is provided on the RAM 13 to the non-volatile memory 14 of the object detection device 1. A classification parameter storage unit 220, which is an area in which machine learning processing parameters of a classifier that classifies image features into object areas and exclusion areas, is pre-stored, is provided on the RAM 103 to the non-volatile memory 104 of the machine learning device 100.

[0043] The image acquisition unit 110, feature extraction unit 125, feature exclusion unit 160, feature matching unit 170, and output unit 180 of this embodiment have the same functions as the image acquisition unit 110, feature extraction unit 125, area classification unit 155, feature matching unit 170, and output unit 180 of the second embodiment.

[0044] The threshold acquisition unit 190 acquires each threshold used in the region classification unit 155. Examples of thresholds used in the region classification unit 155 include a threshold that serves as an index of a target feature indicating the possibility that the feature is a target feature, and a threshold that serves as an index of an excluded feature indicating the possibility that the feature is an excluded feature. The threshold acquisition unit 190 may acquire these thresholds via a predetermined user interface displayed on the display device 70, for example. Alternatively, the thresholds may be acquired from a higher-level computer such as the fog computer 6 or the cloud server 7.

[0045] The region classification unit 155 according to this embodiment classifies at least the regions to be excluded from the image features by classifying the image features for each pixel input from the feature extraction unit 125. When classifying each region, the region classification unit 155 uses a threshold value that serves as an index of the target feature and a threshold value that serves as an index of the excluded feature, both of which are acquired by the threshold acquisition unit 190.

[0046] The object detection device 1 of this embodiment, which has the above-mentioned configuration, when excluding unnecessary image features from the image to be detected, can, for example, display a user interface to the operator, allowing the operator to input a threshold value that will be the target feature index and a threshold value that will be the excluded feature index, and then have the operator confirm the range to be excluded, thereby making it possible to adjust the threshold value so that the excluded area can be appropriately classified.

[0047] [Correction Based on Rule 91 12.09.2024] [Other Embodiments] In the first embodiment described above, an object detection device 1 having a configuration for training a classifier was described. Furthermore, in the second embodiment, an example was described in which an object is detected from an image using a classifier trained by the object detection device 1 of the first embodiment. However, the functions of the object detection device 1 of the first embodiment and the object detection device 2 of the second embodiment may be configured within a single object detection device 1. With this configuration, classifier training and matching processing using the classifier can be performed within a single object detection device 1.

[0048] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the invention or the idea and intent of the present disclosure derived from the content described in the claims and their equivalents. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0049] The following are supplementary notes related to embodiments of the present disclosure. (Supplementary Note 1) An object detection device (1) according to one aspect of the present disclosure includes an image acquisition unit (110) that acquires a first image capturing an object, a feature extraction unit (125) that extracts image features from the first image, a region classification unit (155) that classifies, as an exclusion region, a region in the image features extracted by the feature extraction unit (125) in which an exclusion feature appears, using a classifier that classifies a region in the image features in which a matching feature appears and a region in which an exclusion feature appears based on a matching feature indicating a feature of an image region to be the target of a matching process and an exclusion feature that is a feature of an image region to be excluded from the target of the matching process, a feature exclusion unit (160) that creates a target feature by excluding the image features in the exclusion region from the image features extracted by the feature extraction unit (125), a feature matching unit (170) that performs a matching process between the target feature and the matching feature, and an output unit (180) that outputs a result of the matching process by the feature matching unit (170).

[0050] [Correction based on Rule 91 12.09.2024] (Supplementary Note 2) In another aspect of the object detection device (1) of the present disclosure, the image acquisition unit (110) further acquires a second image in which an object to be excluded from the matching process is captured, the feature extraction unit (125) extracts image features from the first image and the second image, and an area designation receiving unit (120) receives designation of at least one object area in the first image in which an object is captured and at least one exclusion area in the second image in which an object is to be excluded from the matching process, and a feature extraction unit (125) extracts image features from the first image in which an object is captured and at least one exclusion area in the second image in which an object is captured. The image processing system further includes a matching feature acquisition unit (130) that acquires at least one matching feature that is an image feature of the object based on the region; an exclusion feature acquisition unit (140) that acquires at least one exclusion feature that is an image feature of the exclusion region based on the image features extracted from the second image by the feature extraction unit (125) and the at least one exclusion region accepted by the region designation acceptance unit (120); and a learning unit (150) that generates a classifier that classifies the object region and the exclusion region within the image features based on the image features extracted by the feature extraction unit (125), the at least one matching feature, and the at least one exclusion feature.

[0051] (Supplementary Note 3) The area designation receiving unit (120) included in the object detection device (1) according to another aspect of the present disclosure receives a plurality of the exclusion areas. (Supplementary Note 4) The learning unit (150) included in the object detection device (1) according to another aspect of the present disclosure generates a classifier that classifies the plurality of exclusion areas into different categories. (Supplementary Note 5) The image acquisition unit (110) included in the object detection device (1) according to another aspect of the present disclosure acquires a plurality of the second images.

[0052] (Supplementary Note 6) The image acquisition unit (110) included in the object detection device (1) according to another aspect of the present disclosure acquires the plurality of first images, the feature extraction unit (125) extracts image features from the plurality of first images, and the area designation receiving unit (120) receives designation of an object area in the plurality of first images. (Supplementary Note 7) The object detection device (1) according to another aspect of the present disclosure further includes a threshold acquisition unit (190) that acquires at least one of an object feature index indicating a possibility that the feature is the object feature and an excluded feature index indicating a possibility that the feature is the excluded feature, and the area classification unit (155) classifies an area in which the excluded feature appears as an excluded area based on the threshold accepted by the threshold acquisition unit (190).

[0053] A computer-readable recording medium according to one aspect of the present disclosure records a program for operating a computer as an image acquisition unit (110) that acquires a first image of an object, a feature extraction unit (125) that extracts image features from the first image, a region classification unit (155) that classifies, as an exclusion region, a region in the image features extracted by the feature extraction unit (125) in which an exclusion feature appears, using a classifier that classifies regions in the image features in which a matching feature appears and a region in which an exclusion feature appears based on matching features that indicate characteristics of an image region that should be the target of the matching process and exclusion features that are characteristics of an image region that should be excluded from the target of the matching process, a feature exclusion unit (160) that creates a target feature by excluding image features in the exclusion region from the image features extracted by the feature extraction unit (125), a feature matching unit (170) that performs matching processing between the target feature and the matching feature, and an output unit (180) that outputs a result of the matching processing by the feature matching unit (170).

[0054] REFERENCE SIGNS LIST 1 Object detection device 3 Industrial machine 4 Imaging sensor 5 Network 6 Fog computer 7 Cloud server 11 CPU 12 ROM 13 RAM 14 Non-volatile memory 15 Interface 17, 18, 20, 21 Interface 22 Bus 70 Display device 71 Input device 72 External device 100 Machine learning machine 101 Processor 102 ROM 103 RAM 104 Non-volatile memory 110 Image acquisition unit 120 Area designation reception unit 125 Feature extraction unit 130 Matching feature acquisition unit 140 Exclusion feature acquisition unit 150 Learning unit 155 Area classification unit 160 Feature exclusion unit 170 Feature matching unit 180 Output unit 210 Matching feature storage unit 220 Classification parameter storage unit

Claims

1. An object detection device comprising: an image acquisition unit that acquires a first image of an object; a feature extraction unit that extracts image features from the first image; a region classification unit that classifies, as an exclusion region, a region in the image features extracted by the feature extraction unit in which an exclusion feature appears, using a classifier that classifies regions in the image features in which a matching feature appears and a region in which an exclusion feature appears based on a matching feature that indicates the characteristics of an image region that should be the subject of the matching process and an exclusion feature that is the characteristic of an image region that should be excluded from the subject of the matching process; a feature exclusion unit that creates a target feature by excluding image features in the exclusion region from the image features extracted by the feature extraction unit; a feature matching unit that performs matching processing between the target feature and the matching feature; and an output unit that outputs a result of the matching processing by the feature matching unit.

2. The object detection device of claim 1, further comprising: an area designation receiving unit that receives designation of at least one object area in the first image that contains an object and at least one exclusion area in the second image that is to be excluded from the matching process; a matching feature acquisition unit that acquires at least one matching feature that is an image feature of the object based on the image features extracted from the first image by the feature extraction unit and the at least one object area accepted by the area designation receiving unit; an exclusion feature acquisition unit that acquires at least one exclusion feature that is an image feature of the exclusion area based on the image features extracted from the second image by the feature extraction unit and the at least one exclusion area accepted by the area designation receiving unit; and a learning unit that generates a classifier that classifies the object area and the exclusion area in the image features based on the image features extracted by the feature extraction unit, at least one matching feature, and at least one exclusion feature.

3. The object detection device according to claim 2, wherein the area designation receiving unit receives a plurality of the exclusion areas.

4. The object detection device according to claim 3, wherein the learning unit generates a classifier that classifies the plurality of exclusion regions into different categories.

5. The object detection device according to claim 2, wherein the image acquisition unit acquires a plurality of the second images.

6. The object detection device of claim 2, wherein the image acquisition unit acquires a plurality of the first images, the feature extraction unit extracts image features from the plurality of the first images, and the area designation receiving unit accepts designation of object areas in the plurality of the first images.

7. The object detection device of claim 1, further comprising a threshold acquisition unit that acquires at least one of a target feature index indicating a possibility that the feature is a target feature and an excluded feature index indicating a possibility that the feature is an excluded feature, and the area classification unit classifies an area in which the excluded feature appears as an excluded area based on the threshold accepted by the threshold acquisition unit.

8. A computer-readable recording medium having recorded thereon a program for causing a computer to operate as: an image acquisition unit that acquires a first image of an object; a feature extraction unit that extracts image features from the first image; a region classification unit that classifies a region in the image features extracted by the feature extraction unit in which an exclusion feature appears as an exclusion region using a classifier that classifies regions in the image features in which a matching feature appears and a region in which an exclusion feature appears based on a matching feature that indicates the characteristics of an image region that should be the subject of the matching process and an exclusion feature that is a characteristic of an image region that should be excluded from the subject of the matching process; a feature exclusion unit that creates a target feature by excluding the image features in the exclusion region from the image features extracted by the feature extraction unit; a feature matching unit that performs matching processing between the target feature and the matching feature; and an output unit that outputs the result of the matching processing by the feature matching unit.

Images