Image processing apparatus

Abstract

An image processing apparatus of one embodiment of the present disclosure in which detection parameters can be easily set includes a model storage unit that stores a model pattern, a feature point extraction unit that extracts a feature point from a captured image, a raw degree of coincidence calculation unit that calculates a degree of coincidence of the model pattern and the arrangement of the feature point, an object detection unit that detects an object in the captured image by comparing the degree of coincidence with a detection threshold value, a parameter setting unit that sets a detection parameter including at least the detection threshold value, a detection information storage unit that stores detection information including at least the position of the feature point for the feature point of the object detected by the object detection unit, and a simple degree of coincidence calculation unit that calculates a degree of coincidence based on the changed detection parameter and the detection information stored in the detection information storage unit in the case where the detection parameter has been changed.

Description

Technical Field

[0001] This invention relates to an image processing apparatus. Background Technology

[0002] In systems that pick up randomly supplied objects, image processing techniques are widely used to determine the location of objects by analyzing images captured by an imaging device (camera). In such image processing techniques, feature points are generally extracted from the photographic image, and the presence of an object is determined when the pattern of the extracted feature points matches a pre-stored model pattern—that is, when the matching degree is above a threshold.

[0003] If the consistency threshold is too low, false detections will occur, misidentifying objects other than the expected object as the expected object. Conversely, if the consistency threshold is too high, undetected objects will occur. Therefore, users must verify the detection results across multiple images while setting the consistency threshold to minimize false detections and undetected objects.

[0004] The brightness gradient of a photographic image may vary depending on the surrounding brightness. Therefore, a scheme has been proposed that statistically processes the distribution of the brightness gradient of a photographic image and sets a threshold for extracting the brightness gradient of feature points based on its mean or standard deviation (see, for example, Patent Document 1).

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2017-91079 Summary of the Invention

[0008] The problem the invention aims to solve

[0009] Setting multiple detection parameters using thresholds for consistency and brightness gradients is a complex and time-consuming task. Furthermore, while determining the thresholds for feature point extraction statistically is relatively straightforward for user judgment, changing the detection parameters necessitates re-performing the statistical processing and feature point extraction before recalculating consistency, thus requiring time for parameter setting. Therefore, a technique that allows for easy setting of detection parameters for image processing devices is desired.

[0010] Solution for solving the problem

[0011] One aspect of the image processing apparatus disclosed herein is an image processing apparatus for detecting objects in photographed images captured by an imaging device, comprising: a model storage unit that stores model patterns; a feature point extraction unit that extracts feature points from the photographed image; an original consistency calculation unit that calculates the consistency between the model pattern and the configuration of the feature points; an object detection unit that detects the object in the photographed image by comparing the consistency with a detection threshold; a parameter setting unit that sets detection parameters that include at least the detection threshold; a detection information storage unit that stores detection information, including at least the positions of the feature points, for the feature points of the object detected by the object detection unit; and a simplified consistency calculation unit that calculates the consistency based on the changed detection parameters and the detection information stored in the detection information storage unit when the detection parameters have been changed.

[0012] The effects of the invention

[0013] According to this disclosure, an image processing apparatus that allows for easy setting of detection parameters can be provided. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating the image processing structure of the image processing apparatus according to the first embodiment of the present disclosure.

[0015] Figure 2 This is a block diagram illustrating the structure of the image processing apparatus according to the second embodiment of the present disclosure.

[0016] Figure 3 This is an example. Figure 2 A diagram showing the display screen of the user interface section of the image processing device. Detailed Implementation

[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Figure 1 This is a block diagram illustrating the structure of the image processing apparatus 1 according to the first embodiment of the present disclosure.

[0018] Image processing device 1 acquires photographic image data from imaging device C and detects objects in the photographic image. Imaging device C acquires an image of the subject within a predetermined field of view. Imaging device C can be a visual sensor that acquires visible light images, or it can be a sensor that acquires infrared images, distance images, point cloud images, etc.

[0019] The image processing apparatus 1 includes a model storage unit 11, a feature point extraction unit 12, a feature point exclusion unit 13, an original consistency calculation unit 14, an object detection unit 15, a parameter setting unit 16, a detection information storage unit 17, a simplified consistency calculation unit 18, and a parameter modification unit 19.

[0020] The image processing apparatus 1 can be implemented by executing appropriate programs on one or more computer devices having memory, CPU, and input / output interfaces. The various constituent elements of the image processing apparatus 1 described above are constituent elements categorized by the functions of the image processing apparatus 1, and may not be constituent elements that can be clearly distinguished in the physical structure and program structure.

[0021] The model storage unit 11 stores the model pattern obtained by digitizing the outline data of the object. For example, the model pattern can be generated theoretically based on the shape data of the object, or it can be generated based on the outline data of the object, the coordinates of feature points, etc., determined from an image of the object that has actually been photographed.

[0022] The feature point extraction unit 12 extracts feature points from the photographic image that exhibit discontinuous changes with adjacent image regions and may be the outline of an object. Specifically, the feature point extraction unit 12 may be configured to extract points whose pixel attribute values ​​are set to or above the extraction threshold of the parameter setting unit 16 as feature points. As attribute values ​​used to determine feature points, it is desirable to include at least one of the following: color, brightness, magnitude of brightness gradient, and direction of brightness gradient. Furthermore, brightness gradient refers to the difference in brightness between adjacent pixels or unit regions, that is, the rate of change of brightness relative to position, and the direction of brightness gradient refers to the direction in which the change in brightness between adjacent pixels or unit regions is greatest. By setting such attribute values ​​as pointers, points that are highly likely to be the outlines of certain objects or patterns can be extracted from the image. In particular, points with large brightness gradients are more likely to be the outlines of objects.

[0023] The feature point exclusion unit 13 excludes feature points whose distance from the model pattern exceeds a distance threshold from the calculation objects of the original consistency calculation unit 14. In this way, by eliminating noise that obviously does not constitute the outline of the object, the detection accuracy of the object can be improved.

[0024] The original consistency calculation unit 14 calculates the consistency between the model pattern and the configuration of feature points. Specifically, the model pattern is configured in a way that minimizes the sum of the distances or the squares of the distances from all feature points (excluding those excluded by the feature point exclusion unit 13) to the model pattern, and the degree of consistency between the feature points and the model pattern is numerically quantified. The consistency calculation method can be set to be the same as in conventional image processing devices.

[0025] The object detection unit 15 detects objects in the photographic image by comparing the consistency calculated by the original consistency calculation unit 14 with the detection threshold set in the parameter setting unit. That is, if the consistency is above the detection threshold, the object detection unit 15 determines that the feature point is the outline point of the object, and if the consistency is below the detection threshold, it determines that no object exists and does not detect it.

[0026] The parameter setting unit 16 sets detection parameters that include at least the detection thresholds used by the object detection unit 15. That is, the parameter setting unit 16 performs initial settings for the detection parameters and stores the values ​​of the initially set detection parameters or the values ​​of the detection parameters changed by the parameter modification unit 19 after the initial settings. In this embodiment, the parameter setting unit 16 sets, in addition to the distance threshold, the extraction threshold used by the feature point extraction unit 12 and the detection threshold used by the object detection unit 15. The parameter setting unit 16 can be configured to set the initial values ​​of the detection parameters to predetermined values, or it can be configured to wait for user input and set the initial values ​​to values ​​input by the user.

[0027] Preferably, the initial value of the extraction threshold set by the parameter setting unit 16 is a sufficiently low value or a sufficiently wide range, that is, a value set to extract a greater number of feature points than an appropriate number. This allows the detection information storage unit 17 to store detection information for a sufficient number of feature points, enabling the simple consistency calculation unit 18 to more accurately calculate the consistency after the detection parameters have been changed. Alternatively, the parameter setting unit 16 can use captured images to allow the feature point extraction unit 12 to perform feature point extraction trials and determine a value that allows for the extraction of a fixed number of feature points, using this as the initial value of the extraction threshold.

[0028] The detection information storage unit 17 stores detection information, including at least the positions of the feature points, for the feature points of the object extracted by the feature point extraction unit 12. In order to enable the simplified consistency calculation unit 18 to recalculate the consistency for changes in the extraction threshold, the detection information stored in the detection information storage unit 17 preferably further includes the attribute values ​​of the feature points to be compared with the extraction threshold.

[0029] The simplified consistency calculation unit 18 calculates the consistency degree based on the changed detection parameters and the detection information stored in the detection information storage unit 17, even after the detection parameters have been changed by the parameter change unit 19. Thus, by having a simplified consistency calculation unit 18 that recalculates the consistency degree even after the detection parameters have been changed, it is easy to determine whether the change in detection parameters is appropriate. Furthermore, since the simplified consistency calculation unit 18 calculates the consistency degree based on the detection information, it can confirm the consistency degree after the detection parameter change without performing computationally intensive feature point extraction, enabling optimization of the detection parameters in a short time.

[0030] The parameter modification unit 19 modifies the detection parameters to increase the consistency calculated by the simplified consistency calculation unit 18 based on the detection information. In other words, the parameter modification unit 19 automatically optimizes the detection parameters to reduce the number of undetected items.

[0031] Specifically, the parameter changing unit 19 can be configured to temporarily change the detection parameters and cause the simplified consistency calculation unit 18 to recalculate the consistency, and after confirming the value of the detection parameter that makes the consistency sufficiently large, determine the value of the detection parameters. Alternatively, the parameter changing unit 19 can be configured to change the detection parameters to multiple pre-set values ​​to confirm the consistency, and finally select the value that maximizes the consistency. Furthermore, the parameter changing unit 19 can also be configured to use any algorithm such as binary search or mountain climbing algorithm to explore the value of the detection parameters that maximizes the consistency.

[0032] Thus, in the image processing apparatus 1, the parameter changing unit 19 uses the detection information already stored in the detection information storage unit 17 to recalculate the consistency of the simplified consistency calculation unit 18, and optimizes the detection parameters without repeating the feature point extraction performed by the feature point extraction unit 12, which has a large image processing load. Therefore, the detection parameters can be quickly set to the optimal value.

[0033] Figure 2 This is a block diagram illustrating the structure of the image processing apparatus 1A according to the second embodiment of this disclosure. Furthermore, regarding... Figure 2 Image processing apparatus 1A, for use with Figure 1 The same components as the image processing device 1 may be accompanied by the same reference numerals, and repeated descriptions may be omitted.

[0034] The image processing apparatus 1A acquires photographic image data from the imaging device C and detects objects in the photographic image. Furthermore, the image processing apparatus 1A receives input from the user via the input device E and displays information to the user using the display device D. In this embodiment, the image processing apparatus 1A allows the user to actively set the values ​​of the detection parameters.

[0035] As an input device E, one or more devices such as a keyboard or mouse can be used. As a display device D, an image display device such as a CRT, LCD panel, or OLED panel, which has the ability to display photographic images from the imaging device C, can be used.

[0036] The image processing device 1A includes a model storage unit 11, a feature point extraction unit 12, a feature point exclusion unit 13, an original consistency calculation unit 14, an object detection unit 15, a parameter setting unit 16A, a detection information storage unit 17, a simplified consistency calculation unit 18, a parameter modification unit 19A, and a user interface unit 20.

[0037] The parameter setting unit 16A sets the detection parameters according to the input made by the user through the user interface unit 20. In addition, the parameter setting unit 16A may also be configured to set the initial value of the detection parameters based on a preset initial setting value or a past setting value.

[0038] The parameter change unit 19A changes the detection parameters according to the input made by the user through the user interface unit 20.

[0039] The user interface unit 20 can, as needed, individually or in combination, display on the display device D the photographic image taken by the imaging device C, the consistency calculated by the original consistency calculation unit 14, the detection results of the object detection unit 15, the detection parameters set in the parameter setting unit 16A, and the consistency calculated by the simplified consistency calculation unit 18, etc. In addition, in conjunction with these displays, the user interface unit 20 can display on the display device D images such as text boxes that prompt the user to input detection parameter settings, etc., via the input device E.

[0040] Figure 3 This example illustrates the display screen of the display device D accessed via the user interface unit 20. This display screen includes an area for displaying the photographic image (upper left section), an area for displaying detection parameters set by the user (middle right section), and an area for displaying the calculation results of the feature point extraction unit 12, the original consistency calculation unit 14, and the simplified consistency calculation unit 18 (lower section). In this display screen, consistency is displayed as a "score," the detection threshold is displayed as a "score threshold," and the detection threshold related to brightness gradients is displayed as a "contrast threshold." Furthermore, in the photographic image, feature points extracted by the feature point extraction unit 12 are distinguished by color from feature points excluded by the feature point exclusion unit 13 and are overlaid.

[0041] In the image processing apparatus 1A, if the user corrects the detection parameters on the screen, since the feature point extraction unit 12 does not perform feature point extraction, the simplified consistency calculation unit 18 uses the detection information stored in the detection information storage unit 17 to recalculate the consistency. Therefore, the color of the feature points and the consistency (score) value will change without time lag. Thus, the user can not only confirm the consistency value, but also the correspondence between the photographed image and the feature points used in the consistency calculation, and can set more appropriate detection parameters.

[0042] While embodiments of this disclosure have been described above, the present invention is not limited to these embodiments. Furthermore, the effects described in the embodiments are merely examples of the best effects produced by the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0043] As an example, in the image processing apparatus of this disclosure, the parameter changing unit can also switchably combine the functions of automatically changing detection parameters as in the first embodiment and changing detection parameters according to user input as in the second embodiment.

[0044] Explanation of reference numerals in the attached figures

[0045] 1, 1A: Image processing device; 11: Model storage unit; 12: Feature point extraction unit; 13: Feature point exclusion unit; 14: Original consistency calculation unit; 15: Object detection unit; 16, 16A: Parameter setting unit; 17: Detection information storage unit; 18: Simplified consistency calculation unit; 19, 19A: Parameter changing unit; 20: User interface unit; C: Imaging device; D: Display device; E: Input device.

Claims

1. An image processing apparatus for detecting objects in a photographed image taken by an imaging device, the image processing apparatus comprising: The model storage unit stores model patterns. The feature point extraction unit extracts feature points from the photographic image; The original consistency calculation unit calculates the consistency between the model pattern and the configuration of the feature points; An object detection unit detects the object in the photographic image by comparing the consistency with a detection threshold; The parameter setting unit sets at least the detection parameters of the detection threshold; The detection information storage unit stores detection information, including at least the location of the feature points, for the feature points of the object detected by the object detection unit. as well as The simplified consistency calculation unit calculates the consistency degree based on the changed detection parameters and the detection information stored in the detection information storage unit, when the detection parameters have been changed. The feature point extraction unit extracts points whose attribute values ​​are above the extraction threshold as the feature points. The detection parameters set in the parameter setting section include the extraction threshold. The initial value of the extraction threshold is the value of extracting a fixed number or more of the feature points.

2. The image processing apparatus according to claim 1, wherein, The detection information stored in the detection information storage unit includes the attribute value.

3. The image processing apparatus according to claim 2, wherein, The attribute value includes at least one of the feature point's color, brightness, the magnitude of the brightness gradient, and the direction of the brightness gradient.

4. The image processing apparatus according to any one of claims 1 to 3, It also includes a parameter changing unit, which changes the detection parameters to increase the consistency calculated by the simplified consistency calculation unit based on the detection information.

5. The image processing apparatus according to any one of claims 1 to 3, It also includes a user interface section that displays the consistency degree calculated by the simplified consistency degree calculation section.

6. The image processing apparatus according to claim 4, It also includes a user interface section that displays the consistency degree calculated by the simplified consistency degree calculation section.

Images