Image Processing Device

Abstract

An image processing device according to one aspect of the present disclosure for which detection parameters can easily be set comprises: a model storage unit that stores a model pattern; a characteristic point extraction unit that extracts characteristic points from a photographed image; an original matching degree calculation unit that calculates the degree of matching between the model pattern and arrangement of the characteristic points; an object detection unit that detects an object in the photographed image by comparing the degree of matching with a detection threshold value; a parameter setting unit that sets detection parameters including at least the detection threshold value; a detection information storage unit that stores, for characteristic points of the object detected by the object detection unit, detection information including at least the positions of the characteristic points; and a simple matching degree calculation unit that calculates, if the detection parameters are changed, the degree of matching on the basis of the changed detection parameters and the detection information stored in the detection information storage unit.

Description

[Technical field]

[0001] The present invention relates to an image processing device. [Background technology]

[0002] For example, in a system for picking up randomly supplied objects, image processing techniques are widely used to identify the position of an object by analyzing an image captured by an imaging device (camera). In such image processing techniques, generally, feature points in a captured image are extracted, and if there is a high degree of match between the arrangement pattern of the extracted feature points and a pre-stored model pattern, that is, if the numerical value of the degree of match is equal to or greater than a threshold value, it is determined that an object is present.

[0003] If the matching threshold is too low, an erroneous detection occurs in which an object other than the intended object is mistakenly detected as the object. Conversely, if the matching threshold is too high, an undetected object occurs in which the intended object cannot be detected as the object. For this reason, the user needs to set the matching threshold so as to reduce erroneous detection and undetection while checking the detection results in multiple captured images.

[0004] The luminance gradient of a captured image may change depending on the surrounding brightness, etc. For this reason, it has been proposed to statistically process the distribution of the luminance gradient of a captured image and set a luminance gradient threshold value for extracting feature points based on the average value and standard deviation (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2017-91079 A Summary of the Invention [Problem to be solved by the invention]

[0006] When multiple detection parameters such as the threshold for the degree of coincidence and the threshold for the brightness gradient are set, optimizing them is a very cumbersome task. In addition, when the threshold for extracting feature points is determined statistically, the user can easily make a decision in the setting task, but when the detection parameters are changed, it is necessary to perform statistical processing again to re-extract feature points and then calculate the degree of coincidence, so the setting task of the detection parameters takes time. For this reason, a technology that can easily set the detection parameters of an image processing device is desired. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present disclosure is an image processing device that detects an object in an image captured by an imaging device, and includes: a model memory unit that stores a model pattern; a feature point extraction unit that extracts feature points from the captured image; an original match calculation unit that calculates a match between the model pattern and an arrangement of the feature points; an object detection unit that detects the object in the captured image by comparing the match with a detection threshold; a parameter setting unit that sets detection parameters including at least the detection threshold; a detection information storage unit that stores detection information including at least positions of the feature points of the object detected by the object detection unit; and a simplified match calculation unit that calculates the match based on the changed detection parameters and the detection information stored in the detection information storage unit when the detection parameters are changed. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide an image processing device that allows detection parameters to be easily set. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an image processing apparatus including an image processing device according to a first embodiment of the present disclosure. [Diagram 2] FIG. 11 is a block diagram showing a configuration of an image processing device according to a second embodiment of the present disclosure. [Diagram 3]3 is a diagram illustrating an example of a display screen of a user interface unit of the image processing device in FIG. 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Fig. 1 is a block diagram showing a configuration of an image processing device 1 according to a first embodiment of the present disclosure.

[0011] The image processing device 1 acquires captured image data from the imaging device C and detects objects in the captured image. The imaging device C acquires images of subjects within a predetermined field of view. The imaging device C may be a visual sensor that acquires visible light images, or a sensor that acquires infrared images, distance images, point cloud images, etc.

[0012] The image processing device 1 includes a model memory unit 11, a feature point extraction unit 12, a feature point exclusion unit 13, an original match calculation unit 14, an object detection unit 15, a parameter setting unit 16, a detection information memory unit 17, a simplified match calculation unit 18, and a parameter change unit 19.

[0013] The image processing device 1 can be realized by executing an appropriate program on one or more computers having a memory, a CPU, an input / output interface, etc. Each component of the image processing device 1 described above is a classification of the functions of the image processing device 1, and does not necessarily have to be clearly distinguishable in terms of physical configuration and program configuration.

[0014] The model storage unit 11 stores a model pattern in which the contour of an object is digitized. For example, the model pattern may be generated theoretically from shape data of the object, or may be generated from contour data of the object, coordinates of feature points, etc., identified in an image of an actual photograph of the object.

[0015] The feature point extraction unit 12 extracts feature points from the captured image, which are points that change discontinuously from adjacent image regions and may be the contour of an object. Specifically, the feature point extraction unit 12 may be configured to extract points whose pixel attribute values ​​are equal to or greater than an extraction threshold value set in the parameter setting unit 16 or within an extraction range as feature points. The attribute values ​​for determining feature points desirably include at least one of the color, luminance, magnitude of luminance gradient, and direction of luminance gradient of the feature points. Note that the luminance gradient means the difference in luminance with adjacent pixels or unit regions, that is, the rate of change in luminance with respect to position, and the direction of the luminance gradient means the direction in which the amount of change in luminance with adjacent pixels or unit regions is maximum. By using such attribute values ​​as indices, points that are likely to be the contour of some object or pattern in the image can be extracted. In particular, points with a large luminance gradient are likely to be the contour of an object.

[0016] A feature point exclusion unit 13 excludes feature points whose distance from the model pattern exceeds a distance threshold from the calculation targets of the original match calculation unit 14. In this way, by eliminating noise that clearly does not constitute the contour of the object, it is possible to improve the detection accuracy of the object.

[0017] The original matching calculation unit 14 calculates the matching between the model pattern and the arrangement of feature points. As a specific example, the model pattern is arranged so that the sum of the distances or the squares of the distances to the model pattern of all feature points excluding those excluded by the feature point exclusion unit 13 is minimized, and the degree to which the feature points match the model pattern is quantified. The method of calculating the matching can be the same as that of conventional image processing devices.

[0018] The object detection unit 15 detects an object in a captured image by comparing the degree of coincidence calculated by the original coincidence calculation unit 14 with a detection threshold set in the parameter setting unit. That is, the object detection unit 15 determines that the feature points are contour points of an object when the degree of coincidence is equal to or greater than the detection threshold, and determines that no object exists and is not detected when the degree of coincidence is less than the detection threshold.

[0019] The parameter setting unit 16 sets detection parameters including at least a detection threshold used by the object detection unit 15. That is, the parameter setting unit 16 performs initial setting of the detection parameters and stores the initially set values ​​of the detection parameters or the values ​​of the detection parameters changed by the parameter changing unit 19 after the initial setting. In addition to the distance threshold, the parameter setting unit 16 of this embodiment also sets an extraction threshold used by the feature point extraction unit 12 and a detection threshold used by the object detection unit 15. The parameter setting unit 16 may be configured to set the initial values ​​of the detection parameters to predetermined values, or may be configured to wait for a user input and then set the initial values ​​to the values ​​input by the user.

[0020] It is preferable to set the initial value of the extraction threshold set by the parameter setting unit 16 to a sufficiently low value or wide range, that is, a value that extracts more than an appropriate number of feature points. This allows detection information of a sufficient number of feature points to be stored in the detection information storage unit 17, and enables the simplified match calculation unit 18 to relatively accurately calculate the match after the detection parameters are changed. Alternatively, the parameter setting unit 16 may cause the feature point extraction unit 12 to attempt to extract feature points using the captured image, and search for a value at which a certain number of feature points are extracted, and set this as the initial value of the extraction threshold.

[0021] The detection information storage unit 17 stores detection information including at least the positions of feature points of the object extracted by the feature point extraction unit 12. In order to enable the simple match calculation unit 18 to recalculate the match in response to a change in the extraction threshold, it is preferable that the detection information stored in the detection information storage unit 17 further includes attribute values ​​of the feature points to be compared with the extraction threshold.

[0022] When the detection parameters are changed by the parameter changing unit 19, the simple matching calculation unit 18 calculates the degree of match based on the changed detection parameters and the detection information stored in the detection information storage unit 17. In this way, by having the simple matching calculation unit 18 that recalculates the degree of match when the detection parameters are changed, it is possible to easily determine whether or not the change of the detection parameters is appropriate. Furthermore, by having the simple matching calculation unit 18 calculate the degree of match from the detection information, it is possible to check the degree of match after the detection parameters are changed without extracting feature points, which involves a high computational load, and therefore it is possible to optimize the detection parameters in a short time.

[0023] The parameter modification unit 19 modifies the detection parameters so as to increase the degree of match calculated based on the detection information by the simplified match calculation unit 18. In other words, the parameter modification unit 19 automatically optimizes the detection parameters so as to reduce undetected results.

[0024] Specifically, the parameter change unit 19 may be configured to temporarily change the detection parameters to cause the simplified matching degree calculation unit 18 to recalculate the matching degree, confirm the detection parameter values ​​at which the matching degree is sufficiently large, and then determine the detection parameter values. The parameter change unit 19 may be configured to change the detection parameters to a plurality of preset values, confirm the matching degree, and finally select the value at which the matching degree is maximized. The parameter change unit 19 may also be configured to search for the detection parameter values ​​that can sufficiently increase the matching degree based on any algorithm such as a binary search method or a hill climbing method.

[0025] In this manner, in the image processing device 1, the parameter changing unit 19 utilizes the simplified matching calculation unit 18, which recalculates the matching degree using the detection information stored in the detection information storage unit 17, to optimize the detection parameters without repeatedly extracting feature points using the feature point extraction unit 12, which has a large image calculation load, and therefore the detection parameters can be quickly set to optimal values.

[0026] Fig. 2 is a block diagram showing a configuration of an image processing device 1A according to a second embodiment of the present disclosure. Note that, in the image processing device 1A in Fig. 2, components similar to those in the image processing device 1 in Fig. 1 are denoted by the same reference numerals, and duplicated explanations may be omitted.

[0027] The image processing device 1A acquires captured image data from an imaging device C and detects objects in the captured image. The image processing device 1A also accepts input from a user via an input device E and displays information to the user using a display device D. The image processing device 1A of this embodiment allows the user to proactively set values ​​of detection parameters.

[0028] As the input device E, any one or more devices such as a keyboard, a mouse, etc. can be used. As the display device D, an image display device capable of displaying an image captured by the imaging device C, such as a CRT, a liquid crystal panel, an organic EL panel, etc. can be used.

[0029] The image processing device 1A includes a model memory unit 11, a feature point extraction unit 12, a feature point exclusion unit 13, an original match calculation unit 14, an object detection unit 15, a parameter setting unit 16A, a detection information memory unit 17, a simplified match calculation unit 18, a parameter change unit 19A, and a user interface unit 20.

[0030] The parameter setting unit 16A sets the detection parameters in accordance with a user's input via the user interface unit 20. The parameter setting unit 16A may be configured to set the initial values ​​of the detection parameters based on preset initial setting values ​​or past setting values.

[0031] The parameter change unit 19A changes the detection parameters in accordance with the user's input via the user interface unit 20.

[0032] The user interface unit 20 displays, on the display device D, the captured image of the imaging device C, the degree of match calculated by the original match calculation unit 14, the detection result by the object detection unit 15, the detection parameters set in the parameter setting unit 16A, the degree of match calculated by the simplified match calculation unit 18, etc., either alone or in combination as necessary. In addition, the user interface unit 20 displays, in conjunction with these displays, images such as text boxes that prompt the user to input settings for the detection parameters, etc., via the input device E on the display device D.

[0033] 3 illustrates an example of a display screen of the display device D operated by the user interface unit 20. This display screen has an area (upper left) for displaying the captured image, an area (middle right) for displaying a display for the user to set detection parameters, and an area (lower) for displaying the calculation results of the feature point extraction unit 12, the original match calculation unit 14, and the simplified match calculation unit 18. This display screen displays the match degree as a "score," the detection threshold as a "score threshold," and the detection threshold for the luminance gradient as a "contrast threshold." In addition, the feature points extracted by the feature point extraction unit 12 are overlaid on the captured image, with those excluded by the feature point exclusion unit 13 and those not excluded being color-coded.

[0034] In the image processing device 1A, if the user modifies the detection parameters on the screen, the simplified matching degree calculation section 18 recalculates the matching degree using the detection information stored in the detection information storage section 17 without extracting the feature points by the feature point extraction section 12, and therefore the color of the feature point and the value of the matching degree (score) change without any time lag. Therefore, the user can check not only the numerical value of the matching degree but also the correspondence relationship between the captured image and the feature points used for the matching degree, and set more appropriate detection parameters.

[0035] Although the embodiments of the present disclosure have been described above, the present invention is not limited to the above-described embodiments. Furthermore, the effects described in the above-described embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the above-described embodiments.

[0036] As an example, in an image processing device according to the present disclosure, the parameter change unit may have both a function of automatically changing detection parameters as in the first embodiment described above and a function of changing detection parameters in accordance with user input as in the second embodiment, in a switchable manner. [Explanation of symbols]

[0037] 1,1A Image processing device 11 Model storage section 12 Feature point extraction section 13 Feature point exclusion part 14 Original matching calculation unit 15 Object detection unit 16,16A Parameter setting section 17 Detection information storage unit 18 Simple match calculation part 19,19A Parameter change section 20 User Interface Section C. Imaging device D Display device E Input Device

Claims

1. An image processing device for detecting an object in an image captured by an imaging device, a model storage unit for storing a model pattern; A feature point extraction unit that extracts feature points from the captured image; a match degree calculation unit that calculates a match degree between the model pattern and an arrangement of the feature points; an object detection unit that detects the object in the captured image by comparing the degree of coincidence with a detection threshold; a parameter setting unit that sets detection parameters including at least the detection threshold; a detection information storage unit configured to store detection information including at least positions of the feature points of the object detected by the object detection unit; a simplified matching degree calculation unit that calculates, when the detection parameters are changed, the matching degree based on the changed detection parameters and the detection information stored in the detection information storage unit; An image processing device comprising:

2. The feature point extraction unit extracts points whose attribute values ​​are equal to or greater than an extraction threshold as the feature points; the detection parameters set in the parameter setting unit include the extraction threshold; The image processing device according to claim 1 , wherein the detection information stored in the detection information storage unit includes the attribute value.

3. The image processing apparatus according to claim 2 , wherein the attribute value includes at least one of a color, a luminance, a magnitude of a luminance gradient, and a direction of a luminance gradient of the feature point.

4. The image processing device according to claim 1 , further comprising a parameter changing section that changes the detection parameter so that the degree of match calculated by the simplified degree-of-match calculation section based on the detection information becomes larger.

5. The image processing device according to claim 1 , further comprising a user interface unit that displays the degree of coincidence calculated by the simplified degree of coincidence calculation unit.

Images