Image processing device, image processing method and image processing program
Patent Information
- Application Number
- DE102017220752
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2017-09-14
- Filing Date
- 2017-11-21
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2037-11-21
AI Technical Summary
Existing image processing methods for detecting eye features, such as the pupil or iris, suffer from inaccuracies due to noise and environmental factors, leading to erroneous detections and increased computational requirements, especially when ambient light affects eyelid and eyebrow features, and the Hough transform increases throughput demands.
An image processing apparatus that generates an edge image with gradient magnitude and sign information, using a scanning direction aligned with eye alignment, and calculates probabilities based on positive and negative gradient points to accurately detect the pupil or iris, reducing noise influence and computational load.
Accurately detects the pupil or iris even in noisy environments by minimizing the impact of unnecessary features, improving detection accuracy with reduced computational resources.
Abstract
Description
BACKGROUND OF THE INVENTION Technical field
[0001] The present invention relates to an image processing device, an image processing method and an image processing program and relates in particular to a technique that can be used to estimate eye features, such as a gaze based on an image obtained by recording a person's face. State of the art
[0002] A prior art image processing device is known which processes data from an image obtained by capturing a person's face using a camera to identify eye features (for example, JP-A-2012-190351).
[0003] One goal of a technique disclosed in JP-A-2012-190351 is to accurately measure a pupil area.
[0004] The image processing device disclosed in JP-A-2012-190351 comprises: a second differentiation unit 41, which differentiates an eye area of a facial image in which an eye is present at least in one longitudinal direction of the eye in order to obtain a luminance gradient; a binarization unit 42, which binarizes the luminance gradient of the eye area in order to extract an edge point; and an eyelid outline identification unit 44, which identifies a curve expressed by both endpoints and a control point and which fits the edge point as a curve expressing an outline of an upper eyelid or a lower eyelid, wherein the endpoints are an inner corner of the eye and an outer vertex.
[0005] In a device that estimates a person's gaze by processing a moving image of the person's face, it is desirable to accurately capture a central coordinate of a black part of an eye, i.e., an iris region, or a pupil region in the image.
[0006] For example, in a method for detecting a black part of an eye in image processing, disclosed in JP-A-2012-190351, an edge in an image is used as a recognition feature. In particular, an absolute value obtained by filtering using a differentiation filter, such as the Sobel filter, is used as a feature size.
[0007] However, this method inevitably creates an edge from an object other than the target, for example, through the reflection of light projected onto an eyelid, eyebrow, or eyeball. Therefore, a deterioration in detection accuracy or even a detection error can occur. In a case where the detection position of a black part of an eye deviates, an error will occur during the estimation of a gaze.
[0008] Additionally, the technique disclosed in JP-A-2012-190351 uses the Hough transform. Therefore, increased throughput is unavoidable for high accuracy, and the computer hardware must have high processing capacity. During the binary or ternarization of image data, a threshold becomes problematic, and the result is easily affected by ambient light during acquisition. In a case where an ellipse acquisition algorithm is used, it is difficult to acquire a target even in a state where an eye is moving sideways and the shape of a black part of the eye is significantly collapsed on a two-dimensional image.
[0009] The invention was made in view of the circumstances described above, and one object of it is to provide an image processing device, an image processing method and an image processing program that are capable of accurately capturing a pupil or an iris. SUMMARY OF THE INVENTION
[0010] To achieve the objective, an image processing device according to the invention comprises the following points (1) to (4). (1) An image processing device comprises: a data processing unit that processes data from a facial image captured to encompass a face, where the data processing unit is: generates an edge image that stores information regarding a gradient size and whether the gradient is positive or negative, by filtering the face image to capture an edge in a scanning direction; in a state in which a sampling curve including an arc-shaped curve virtually positioned on a plane of a three-dimensional model is projected onto the edge image, a sample value is extracted as the information regarding a gradient size and whether the gradient is positive or negative, from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve; a probability with respect to a sampling curve calculated by placing points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the plurality of points forming the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the plurality of points forming the sampling curve; and a scanning curve with a maximum probability as a pupil or an iris captured among a multitude of scanning curves. (2) Image processing device according to (1), wherein the scanning direction in which the data processing unit detects an edge by filtering is a direction in which eyes are oriented in the face image. (3) Image processing device according to (2), wherein the scanning curve has an arc shape and is projected onto the edge image in a state in which a bottom is directed downwards as the most swelling part of the arc in the three-dimensional model. (4) Image processing device according to (1), wherein the data processing unit calculates a probability with respect to the sampling curve by adding gradient sizes of the points with a positive gradient in the first point group and by adding gradient sizes of the points with a negative gradient in the second point group.
[0011] In the image processing device with the configuration described above according to (1), even in a highly noisy environment, adverse effects from unnecessary features that generate noise can be avoided with minimal computational effort, and a pupil or iris can be accurately detected. Generally, when edge information is used to detect an image, an absolute value is used to remove a sign. However, when a detection target is limited to the pupil or iris, detection accuracy can be improved by using not only a gradient magnitude but also a difference between positive and negative signs at each edge of an image to separate the pupil or iris as the detection target from ambient noise (for example, from the projection of light).For example, in a case where an image area of an eyeball is scanned sequentially from left to right, the number of points with a positive gradient in the first point group is high when the iris is positioned to the right of the iris's outline, or when the pupil is positioned to the right of the iris's outline. Conversely, the number of points with a negative gradient in the second point group is high when the iris is positioned to the left of the iris's outline, or when the pupil is positioned to the left. Therefore, the probability is highest when the scan curve's position coincides with the pupil's or iris's outline. In this case, the pupil or iris can be accurately captured.
[0012] In the image processing device with the configuration described above according to (2), information required for accurate iris position detection can be efficiently obtained. Assuming that the scanning direction in which an edge is detected is a vertical direction of the eye, a boundary adjacent to the outline of the eye's iris could be an upper or lower eyelid. Since an eyelid is likely to be affected by ambient light or makeup, it is difficult to detect a boundary with the iris on an edge image. However, by defining the scanning direction in which an edge is detected as a direction in which the eyes are oriented in the facial image, for example, a horizontal direction, there is only a small influence from an upper or lower eyelid. Therefore, the iris detection accuracy is improved.
[0013] In the image processing device with the configuration described above according to (3), information necessary for accurately detecting an iris position can be efficiently obtained. In the eye area of a general facial image, the upper end of the iris borders the upper eyelid, or part of the iris is often covered by the upper eyelid. Therefore, it is difficult to capture the outline of the upper end of the iris. On the other hand, a border with a white part is often clearly visible on the right and left sides and in the lower end of the iris. Therefore, it is easy to capture the outline of the iris. Accordingly, it is easy to capture the outline of the iris by assuming an arc shape in a state where the base, as the most swollen part of the arc, points downwards.
[0014] In the image processing device with the configuration described above according to (4), a feature of an edge of a boundary from the iris to the white part of the eye or a boundary from the pupil to the iris in a scanning direction is captured as a gradient magnitude of the points with a positive gradient, and a feature of an edge of a boundary from the white part to the iris or a boundary from the iris to the pupil in the scanning direction is captured as a gradient magnitude of the points exhibiting a negative gradient. Based on these features, the probability can be calculated.
[0015] To achieve the goal, an image processing method according to the invention comprises the following (5).
[0016] (5) Image processing method using an image processing device with a data processing unit that processes data of a face image taken to encompass a face. The image processing method includes:
[0017] Generating an edge image that stores information regarding a gradient size and whether the gradient is positive or negative, using a filter that captures an edge in the face image in one scanning direction;
[0018] Extracting, in a state in which a sampling curve including an arc-shaped curve virtually positioned on a plane of a three-dimensional model is projected onto the edge image, a sample value as the information regarding a gradient size and whether the gradient is positive or negative, from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve;
[0019] Calculating a probability with respect to a sampling curve by placing points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the multitude of points forming the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the multitude of points forming the sampling curve; and
[0020] Captured on a scanning curve with a maximum probability as a pupil or an iris among a multitude of scanning curves.
[0021] In the image processing method with the configuration described above according to (5), as in the case of the image processing device according to (1), even in a highly disturbed environment, adverse effects of unnecessary features that generate noise can be avoided with a small computational effort, and the pupil or iris can be accurately detected.
[0022] To achieve the goal, an image processing program according to the invention comprises the following (6) or (7).
[0023] (6) Non-transitory computer-readable medium storing a program that causes a computer to execute a process for processing a captured image to encompass a face. The process includes: Generating an edge image that stores information regarding a gradient size and whether the gradient is positive or negative, using a filter that captures an edge in the face image in one scanning direction; Extracting, in a state in which a sampling curve including an arc-shaped curve virtually positioned on a plane of a three-dimensional model is projected onto the edge image, a sample value as the information regarding a gradient size and whether the gradient is positive or negative, from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve; Calculating a probability with respect to a sampling curve by placing points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the multitude of points forming the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the multitude of points forming the sampling curve; and Captured on a scanning curve with a maximum probability as a pupil or an iris among a multitude of scanning curves.
[0024] (7) An image processing program that enables a computer to perform the respective steps of the image processing procedure according to (5).
[0025] By enabling a predetermined computer to execute the image processing program with the configurations described above according to (6) or (7), as in the image processing device according to (1), even in a highly disturbed environment, adverse effects of unnecessary features that generate noise can be avoided with a small computational effort, and the pupil or iris can be accurately detected.
[0026] In the image processing device, image processing method, and image processing program according to the invention, the pupil or iris can be accurately detected. The invention is useful because, even in a highly noisy environment that causes detection errors, the negative influence of unnecessary features that generate noise can be avoided with minimal computational effort.
[0027] The invention is described in simple terms below. Furthermore, the details of the invention are clarified by describing the following embodiments of the invention (hereinafter referred to as "embodiments") with reference to the accompanying drawings. List of characters Fig. Figure 1 is a perspective view that represents a specific example of a state in which a vehicle-mounted image processing device is arranged; Fig. Figure 2 is a flowchart that summarizes the eye-tracking algorithm implemented by the image processing device according to the embodiment of the invention; Fig. Figure 3 is a schematic diagram that represents a relationship between a three-dimensional model of an eyeball and a two-dimensional image; Fig. Figure 4 is a diagram illustrating a specific example of the flow and data of a process for searching for an iris in an image in the state-of-the-art eye-tracking algorithm; Fig. 5 is a flowchart that details the process in Fig. 2 represents step S18; Fig. 6 is a schematic diagram that presents a specific example of data processing in a case where a Fig. 5. The illustrated process is carried out; Fig. Figure 7 is a diagram illustrating how the detection position of an iris deviates due to the state of the art in gaze detection algorithm; Fig. Figure 8A is a diagram that shows an example of a sampling direction of the Sobel filter, capturing an edge of an image and a processed state and an unprocessed state of each data value; Fig. 8B is a diagram that represents an example of processed data from an edge image; Fig. 9 is a flowchart that details the process described in Fig. Step 5, S18d, is illustrated; Fig. Figure 10 is a schematic diagram showing a specific example of a pattern shape of a particle filter used to capture an iris; Figures 11A and 11B are schematic diagrams illustrating specific examples of processing data in a case where a Fig. 9. The illustrated process is carried out; Fig. Figure 12 is a schematic diagram showing a sample shape of a particle filter according to a second embodiment; Figures 13A and 13B are schematic diagrams showing specific examples of processing data in a case where the pattern shape of the in Fig. The particle filter shown in section 12 is used; Fig. Figure 14 is a graph that illustrates an example of a relationship between the application of the invention and the detection error; Fig. Figure 15 is a schematic diagram representing a sample shape of a particle filter according to a third embodiment; and Figures 16A and 16B are schematic diagrams showing specific examples of processing data in a case where the pattern shape of the in Fig. The particle filter shown in section 15 is used. DETAILED DESCRIPTION OF THE EXECUTION FORM
[0028] Specific embodiments relating to an image processing device, an image processing method and an image processing program according to the invention are described below with reference to the drawings. <Spezifisches Beispiel der Umgebung der Erfindung>
[0029] Fig. Figure 1 illustrates a specific example of a state in which an image processing device attached to a vehicle is arranged.
[0030] In the Fig. The example shown is a camera. 12 and a processing unit 13 in a vehicle interior 10 planned. In particular, the camera 12 in a measuring unit or a steering column cover in front of a person 11 provided for, which is seated in a driver's seat, that is, a driver, so that the driver's face and the like, including the eye area, are visible. 11a from the camera 12 be depicted.
[0031] The processing unit 13 processes two-dimensional image data obtained by capturing the person's face 11 with the camera 12 to be obtained, and estimates information regarding the direction of a glance. 11bbased on data from the eye area 11a off. The information about the view 11b , which are processed by the processing unit 13 The estimated gaze can be used by an onboard unit to detect whether the driver is looking towards the safety monitoring system. Naturally, the gaze information can be used for various applications. Accordingly, the image processing device according to the invention is assumed to be, for example, the processing unit. 13 in Fig. 1 or as part of the processing unit 13 It may be configured. <Zusammenfassung des Blickerfassungsalgorithmus>
[0032] A summary of an eye-tracking algorithm implemented by an image processing device according to an embodiment of the invention is described. Fig. Figure 2 is a flowchart summarizing the eye-tracking algorithm implemented by the image processing device according to the embodiment of the invention. By allowing a computer of the image processing device to execute a predetermined program, operations are performed according to the flowchart shown in Figure 2. Fig. The eye-tracking algorithm shown in Figure 2 is executed sequentially.
[0033] The camera 12 It captures images at regular intervals of an area that includes the person's face. 11 It contains and outputs image signals. The computer of the image processing device stores image signals from a frame from the camera. 12 in step S12.
[0034] In the next step, S13, an image data format conversion, including grayscale conversion, i.e., "preprocessing," is performed. For example, two-dimensional ( 2D) arranged image data in which 8-bit data representing the luminance in a gradation range of "0 to 255" are arranged vertically and horizontally at each of the pixel positions in a frame in a scanning direction during recording.
[0035] In the next step, S14, for example, the face is detected using the Viola-Jones method, and an area containing the face is extracted as a rectangular region from the two-dimensional image data of a frame. That is, a difference in the shading of the face is defined as a feature, and a facial region is extracted using a detector prepared using reinforcement learning. The Viola-Jones technique is disclosed, for example, in the following document: P. Viola and MJ Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features,” IEEE CVPR (2001).
[0036] In the next step S15, for example, an eye area is captured using the “Viola-Jones method” from the data of the rectangular facial area extracted in S14.
[0037] Additionally, the embodiment assumes a case where a gaze is captured using a model-based method with a 3D eyeball model described below, and thus it is necessary to identify an eyeball center. This eyeball center is estimated in step S16. Here, a central coordinate of the rectangular eye area captured in step S15 is assumed to be the eyeball center. For example, it is assumed that a method for determining an eyeball center based on the positions of an inner and outer canthus of the eye, or a method for calculating an eyeball center position while estimating a facial outline based on feature points of the face, is used.
[0038] In step S17, a rough search for a pupil or iris is performed by applying a template-fitting procedure to data from the rectangular eye area acquired in step S15. Specifically, a black circle image is positioned as a template to match a binarized image of an eye obtained by slicing the periphery of the eye. A coordinate of the center of the black circle image (the center of the black circle) is estimated to be the most likely midpoint of the pupil or iris in the eye image, and the radius of the black circle image is estimated to be the most likely radius of the pupil or iris in the eye image. The process in step S17 is performed to establish an approximate target for the midpoint and radius of the pupil or iris in the eye image.
[0039] In step S18, a particle filter method is applied, and the central position and radius of the pupil or iris sought in S17 are used to determine the central position and radius of the pupil or iris more accurately. Step S18 comprises the characteristic method of the invention. Therefore, the details of step S18 are described below.
[0040] Through the processes of steps S13 to S18, data on the eyeball center coordinate and data on the coordinate of the midpoint of the pupil or iris can be obtained from an image of a frame, and the numerical data from this is output in step S19. A gaze direction can be identified based on the eyeball center coordinate and the coordinate of the midpoint of the pupil or iris. Additionally, a gaze can be captured in real time by repeating the loop processes of steps S11 to S20 until images are output from the camera. 12 will be stopped. <Beschreibung des 3D-Augapfel-Modells>
[0041] Fig. Figure 3 illustrates a relationship between a three-dimensional ( 3D ) Model of an eyeball and a two-dimensional ( 2D ) Picture.
[0042] In a 3D eyeball model 30, which is in Fig. As shown in Figure 3, the eyeball is a spherical body that is defined by an eyeball center. 31 and a radius R is expressed. Additionally, a circular iris is present. 32 of the eye, present on the surface of the eyeball and a circular pupil 33 is in the middle of the iris 32 available.
[0043] Accordingly, a gaze direction can be identified as a direction that extends from the center of the eyeball. 31 to the center of the iris 32 or the pupil 33 moved, and can be expressed by a yaw and a pitch, and the yaw is an angle of rotation with respect to a reference direction in a horizontal plane, and the pitch is an angle of rotation with respect to a vertical reference direction. In a case where the eyeball center 31 When used as a reference, a central coordinate of the iris can be used. 32 or the pupil 33can be expressed by an eyeball radius R, a yaw rate and a slope.
[0044] On the other side, one of the camera's buttons is pressed. 12 The captured image represents a specific two-dimensional plane. Therefore, in a case where a camera captures an image... 12 recorded two-dimensional image 34 When applied to the 3D eyeball model 30, it is necessary to perform an interconversion between the two-dimensional image and the three-dimensional image. Therefore, for example, the interconversion is performed using the following calculation expression. X = − R × cos ( Steigung ) × sin ( Gierung ) Y = R × sin ( Steigung ) X: the distance from the center of the eyeball 31 on one plane of the two-dimensional image in one x-direction Y: the distance from the center of the eyeball 31 on one plane of the two-dimensional image in a y-direction <Spezifisches Beispiel für den Suchprozess von Iris aus dem Bild>
[0045] The following section describes a specific example of a process for finding the iris from the image. <Beschreibung des Prozesses des Blickerfassungsalgorithmus des Standes der Technik> <prozessfluss>
[0046] Before describing the details of the eye-tracking algorithm implemented by the image processing device according to the embodiment of the invention, the details of a prior art eye-tracking algorithm are described. Fig. Figure 4 is a diagram illustrating a specific example of the flow and data of a process for searching for an iris in an image in the state-of-the-art eye-tracking algorithm.
[0047] In step S41 of Fig. 4. A rectangular area including the eye and the eye's periphery is created from two-dimensional image data of a frame using the result of step S15 of Fig. 2 is cut out, and data D41 is obtained.
[0048] In step S42 of Fig. 4. Data will be 42 D41 is generated by binaryizing the data obtained in S41, so that the pixels have two gradation values of black and white. Next, the template adjustment from step S17 is performed. Fig. 2. This was carried out on the data D42. That is, while the image of the data D42 is scanned using a template which is a black circular image D4t with a shape similar to an iris, an approximate iris position P41 with the most similar feature is sought, and the position and radius or diameter of the iris are identified.
[0049] In step S43 of Fig. In step 4, a Sobel filtering process is performed on the eye data D41 obtained in step S41. Specifically, the eye data D41 is sequentially sampled horizontally from left to right. Black (gradation value: 0) is output for a portion that has no change in luminance, and as the gradient of a change in luminance increases, the output value approaches white (gradation value: 255). In this way, an edge is detected. The result is data D43 of the eye image in which the edge is extracted.
[0050] In step S44 of Fig. 4. A particle filtering process is carried out on the data D43 of the eye image obtained in step S43, using the approximate iris position obtained in step S42 as the starting point. <Verfahren zur Erkennung von Ionen unter Verwendung eines Partikelfilters>
[0051] Fig. 5 illustrates the details of step S44, which is in Fig. 4 is shown. That is, Fig. Figure 5 illustrates the procedure for capturing the iris position using the particle filter. Fig. Figure 6 illustrates a specific example of data processing in a case where the in Fig. The process is carried out as illustrated in section 5.
[0052] By performing the coarse search of step S42 on binary data D91 of Fig. 6 becomes a circular iris area 91 recorded. In step S18a of Fig. 5 will plot a central position of the iris, obtained by the coarse search of step S17 or S42, into a coordinate system (R, Pp, Py) of the 3D eyeball model 30 based on the expressions ( 1 ) and ( 2 ) converted, and an eyeball rotation angle (Pp, Py) representing the midpoint of the iris is obtained. R: an eyeball radius Pp: a variable representing a rotation angle slope Py: a variable representing a rotation angle yaw.
[0053] Accordingly, the position of a circular iris region can be 92 , obtained by the coarse search, are reflected at an eyeball surface position on the 3D eyeball model 30A, as in Fig. 6 shown.
[0054] In step S18b of Fig. 5. Several particles required to detect actual iris position candidates are scattered around the central position (Pp, Py) of the iris obtained by the coarse search. In particular, for example, 100 sets of scanning circuits are used. 93 formed using random numbers. These sampling circuits 93 can be reflected at ocular surface positions on a 3D eyeball model 30B, as in Fig. 6 shown. Each of the scanning circuits 93 is a circular area centered on a position obtained by adding different random number components (Ppd, Pyd) to the central position (Pp, Py) of the iris. Ppd: a random number component added to the rotation angle slope Pyd: a random number component that is added to the yaw angle.
[0055] During the formation of the 100 sets of sampling circuits 93 Each random number component (Ppd, Pyd) is generated to add random noise according to a normal distribution. Accordingly, the sampling circuits are 93 as in the Fig. 6 depicted 3D eyeball model 30B shown, randomly at different positions around the iris area 92 trained around.
[0056] In step S18c of Fig. 5 will be each of the 100 sets of scanning circuits. 93 on a plane of an edge image D92 (corresponding to the data D43 in Fig. 4) projected, which in Fig. Figure 6 shows the shape of the iris area. 92 and each of the scanning circuits 93 is circular. However, if the position of each of the circles is not the same as the front of the camera. 12 Since each circle on the 3D eyeball model 30 is projected onto a plane of a two-dimensional image in a tilted state, the shape of the circle in a two-dimensional plane is elliptical. To simplify the description, the scanning circle projected onto a two-dimensional plane is therefore referred to as... 93 as a scanning ellipse 94 The scanning ellipse is called the scanning ellipse. 94 includes several points that are equally assigned to the respective positions on an arc.
[0057] In step S18d of Fig. 5. A “sampling process” described below is performed. That is, with respect to each of the sampling ellipses 94 The edge thicknesses (gradient sizes) of edge images of points (in this example a total of 120 points) that form the ellipse are added, and the result is expressed as a probability of a sampled ellipse. 94 determined.
[0058] Regarding all 100 sets of sampling ellipses 94 The processes of steps S18c and S18d are repeated. Once all processes are complete, the process proceeds from step S18e to step S18f. In step S18f, one of the 100 sets of sampling ellipses is selected. 94 a scanning ellipse 94 The iris is identified with the maximum probability captured in step S18d, and its position and shape are captured as the actual iris position and shape. The process of capturing the iris using a particle filter in the prior art gaze-capture algorithm has been described above. <Beschreibung der Erfassungspositionsabweichung in einem Blickerfassungsalgorithmus des Standes der Technik>
[0059] Fig. Figure 7 is a diagram illustrating how the detection position of an iris can deviate due to a prior art eye-tracking algorithm. Assuming that the image processing device performing the processes described above is operating, for example, in an outdoor environment, the detection accuracy deteriorates due to ambient light.
[0060] For example, in a case where a binary search S51 (according to steps S17 and S42) is performed on the binaryized data D52, which is based on the in Fig. When 7 eye image data D51 is generated, an iris position P51 may be incorrectly detected. As a result, in a case where the particle filter process S52 described above is applied to an edge image D53, a starting point from which particles are scattered deviates. During probability assessment, edge information other than the iris is likely to be detected. Ultimately, the detection accuracy deteriorates.
[0061] The reason for this is as follows: the upper eyelid is illuminated by sunlight, thus creating a black area in the binarized image. When a binarization template adjustment is performed on this image, it is determined that the iris is positioned on the upper eyelid within the eye image. Scan circles are then formed based on this position. Consequently, the positions of 100 sets of scan circles deviate from the overall position of the upper eyelid. This positional deviation is also significant in the particle filtration process.
[0062] In image data D54, which is in Fig. As shown in Figure 7, an actual iris position is equal to P53, but a position P52 that deviates significantly from P53 is recorded as the result of the particle filtration process S52. Thus, in a case where the iris is captured using the prior art eye-tracking algorithm, the iris capture position can deviate as described above. <Prozess der Ersten Ausführungsform>
[0063] To reduce the detection position deviation of the iris, which is in Fig. As shown in section 7, the image processing device according to the embodiment comprises the following:<Konfiguration A> and<Konfiguration B> <Konfiguration A>
[0064] An edge image, which stores information regarding a gradient size and whether the gradient is positive or negative, is generated by filtering a face image to capture an edge in a sampling direction. For example, in a case where an edge is extracted and the data D43 is stored in the Fig. In step S43, as shown in step 4, a special edge image is generated that stores information regarding a gradient size and a positive or negative sign of the gradient. <Konfiguration B>
[0065] A probability with respect to a sampling curve is calculated by placing points with a positive gradient as probability evaluation targets in a first point group, which is positioned on a front side in the sampling direction, below the multitude of points that form the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group, which is positioned on a back side in the sampling direction below the multitude of points that form the sampling curve. <Spezifisches Beispiel für „Konfiguration A">
[0066] Fig. Figure 8A illustrates an example of a sampling direction of the Sobel filter, which captures an edge of an image and a processed state and an unprocessed state of all data. Fig. Figure 8B is a diagram illustrating an example of processed data from an edge image. In the example shown in Figures 8A and 8B, it is assumed that the Sobel filter position on an image is sampled in a horizontal direction, moving from left to right of the image, or in a direction in which eyes are oriented, to accurately capture a feature from the boundary of an eye region in the left-right direction.
[0067] In the unprocessed data D71 shown in Figs. 8A and 8B, the luminance of an area between a boundary D71a and a boundary D71b in the horizontal direction is assumed to be a black level like in the iris, and the luminance of the other areas is a white level like the white part of the eye.
[0068] Therefore, in a case where the unprocessed data D71, shown in Figures 8A and 8B, is processed using the Sobel filter, edges with a change in luminance are detected at the positions of the respective boundaries D71a and D71b, and processed data D72 is obtained including this edge information. In this case, unlike in a conventional case where the Sobel filter is applied, the processed data D72 stores information not only about the gradient magnitude (edge strength) but also about the positive or negative sign of the gradient.
[0069] This means that in the processed data D72, a black value is stored in a range of "-255 to -1" at the position of edge D72a, a white value is stored in a range of "1 to 255" at the position of edge D72b, and a gray value "0" is stored in the other areas.
[0070] This means that, as in the boundaries D71a and D71b of the unprocessed data D71, the processed data D72 store a positive or negative sign of the gradient such that the boundary of the iris start position, where the luminance (gradation) changes from white to black, and the boundary of the iris end position, where the luminance (gradation) changes from black to white, can be distinguished from each other. In this way, data from an edge image D73, which is in Fig. The data shown in Figure 8B can be obtained. The edge image data D73 are suitable for capturing a feature of a boundary between the iris and a white part of the eye in one scanning direction (horizontal direction in Figures 8A and 8B). <Spezifisches Beispiel für „Konfiguration B">
[0071] Fig. Figure 9 illustrates the detailed procedure, which includes characteristic features of the “sampling process” of step S18d, which is described in Fig. 5 is shown. Fig. Figure 10 illustrates a specific example of a pattern shape for a particle filter used to capture the iris. Additionally, Figures 11A and 11B are schematic diagrams representing specific examples of processing data in a case where a Fig. The process is illustrated in section 9.
[0072] The in Fig. The pattern shown in 10 corresponds to scanning circuits. 93A and scanning ellipses 94A , which are shown in Figs. 11A and 11B, and are used to calculate a suitable probability at a position of each of the sampling ellipses. 94A used in “the sampling process”, which in Fig. 9 is shown. Fig. 11A the same as Fig. Section 6, except that edge image D73 is shown instead of edge image D92, does not repeat the detailed description of it. Fig. 11B the same as Fig. In section 7, except that an edge image D73 is shown instead of the edge image D53, the detailed description of it is not repeated.
[0073] A in Fig. The filter pattern FP0 shown in Figure 10 is formed in a circular (or elliptical) sphere shape, and has several sampling points. 101 are arranged at regular intervals around the circumference. The total number of scanning points 101 For example, in the filter pattern FP0, the value is 120.
[0074] Furthermore, the filter pattern FP0 is divided in the middle into right and left semicircles and comprises a left pattern area FPL, corresponding to the left semicircle, and a right pattern area FPR, corresponding to the right semicircle. Sample points 101 in the left pattern area FPL and sample points 101 In the right-hand sample area FPR, the left group points PGL and the right group points PGR are defined and distinguished from each other.
[0075] In steps S101 to S108 of the "scanning process", which is in Fig. As shown in Figure 9, the following process is carried out sequentially on an “i” number of sampling points. 101 performed, which involved a scanning ellipse 94A form, which is selected.
[0076] In step S102, it is determined whether the corresponding sampling point 101 is to the left of a central coordinate of an ellipse or not, that is, whether the sampling point 101 The left group point is PGL or not. In a case where the sampling point 101 The left group point is PGL, the process proceeds to step S103. In a case where the sampling point 101 Once the right group point PGR is reached, the process proceeds to step S105.
[0077] In step S103, a value from one pixel of the edge image D73, shown in Figs. 11A and 11B, is obtained (sampled) at the position of a sampling point. 101 The currently selected step is determined, and it is then determined whether the sign of the sample is positive or negative. If the sign of the sample is "negative," the process proceeds from step S103 to step S104. If the sign of the sample is "positive," the process proceeds to step S107.
[0078] In step S104, the result of multiplying the current sample value obtained in S103 by "-1" is added to a value of a variable X. That is, in this case, the sampling point 101 The left group point PGL belongs to the second point group. Therefore, the sample value of the corresponding pixel with a negative gradient is set as a probability assessment target. To set the sign of the sample value to "positive" during the calculation, the sample value is multiplied by "-1" in step S104.
[0079] In step S105, a value from one pixel of the edge image D73, shown in Figs. 11A and 11B, is obtained (sampled) at the position of a sampling point. 101 The currently selected step is determined, and it is then determined whether the sign of the sample is positive or negative. If the sign of the sample is "positive," the process proceeds from step S105 to step S106. If the sign of the sample is "negative," the process proceeds to step S107.
[0080] In step S106, the current sample value obtained in S105 is added to a value of the variable X. That is, in this case, the sampling point is 101 The rightmost group point is PGR and belongs to the first point group. Therefore, the sample value of the corresponding pixel with a positive gradient is defined as a probability assessment target.
[0081] In step S107, a value of a variable (i) is assigned, which represents a selection position of a sampling point. 101 represents the value that is processed on the circumference of the ellipse and updated for the next position.
[0082] Once the processes of steps S101 to S108, which are in Fig. 9 are shown, at all sampling points 101 Once completed, the variable X stores the sum of the probability rating values obtained from the respective sampling points. 101 This means that the value of the variable X represents the probability with respect to a sampling ellipse. 94A dar (see Figs. 11A and 11B), which is currently selected.
[0083] By carrying out the in Fig. In the “scanning process” shown in Figure 9, based on the data of edge image D73 generated in “Configuration A”, the features of positions where the positive-negative inversion of the edge thickness occurs on the opposite borders of the iris can be accurately detected. That is, the probability of a scanning ellipse where the edge thickness is positive on the right side of the center and negative on the left side of the center is calculated as high.
[0084] In this way, the processes of steps S101 to S108 are applied to all sampling circuits. 93A performed, and a scanning circuit 93A The iris is recognized with the highest probability. In the image processing device according to the embodiment of the invention, within the boundaries between the white part of the eye and the iris, the number of points with a positive edge thickness is large in a right outline position of the iris, and the number of points with a negative edge thickness is large in a left outline position of the iris. Therefore, the probability of a scanning circuit is 93A The probability of scattering at a position near the outline of the iris is conspicuously high. Consequently, in a highly noisy environment, even in a case where the iris is erroneously determined to be positioned at the upper eyelid in an eye image, the probability of any sampling circle scattered based on the upper eyelid position is extremely low by performing binarization template fitting. Therefore, the sampling circle located near the side of the upper eyelid is excluded from the iris candidates. In this way, the negative influences of unnecessary, noise-generating features can be avoided with a small computational effort, and the iris can be accurately captured.
[0085] As described above, even in a case where the iris is mistakenly determined to be positioned in the position of an upper eyelid in an eye image, it is necessary, by performing the binary template adjustment, to take note of the following point in order to exclude a scanning circle located near the upper eyelid side from iris candidates. That is, in a case where the scanning circles 93A The width of a numerical range of random numbers, which are known as the random number components (Ppd, Pyd) for determining the centers of the respective sampling circuits, is the scattered range. 93A They must be large to be introduced. In a case where the width is small, the sampling circuits are 93A locally scattered. In this case, the multiple sampling circuits 93A scattered in incorrect positions, and it is difficult to improve the recognition accuracy of the iris. Therefore, even in a case where the iris is mistakenly determined to be positioned at the location of an upper eyelid in an eye image, it is desirable to use scanning circles 93A to adjust the total number of random value components and sampling circuits, as in the case where the sampling circuits are scattered in the wrong position, to prevent them from being scattered in the wrong position. 93A are scattered around a suitable position of the iris. By appropriately adjusting the total number of random number components and sampling circuits, the process of step S17, which is described in Fig. 2 is not necessary.
[0086] Additionally, in the Fig. In the “sampling process” shown in 9, it is assumed that the data (D72) of the edge image that is generated while the data is sampled from left to right is as shown in Fig. Figure 8A shows that the process must be edited. In a case where the scanning direction changes during edge image generation, positive-negative inversion of the edge thickness occurs, and the positions of the positive-negative inversion change. Therefore, depending on the change in scanning direction, it is necessary to refer to the details of steps S102 to S106 in [reference missing]. Fig. 9 to change.
[0087] The filter's scanning direction during edge image generation is not limited to the direction in which both eyes are aligned or to the horizontal direction. For example, the scanning direction can be changed to an oblique direction. However, it is preferred that the filter direction in which an edge is detected be a horizontal direction or a direction in which both eyes are aligned, since an edge in a boundary between the iris and the white part of an eye is more likely to be detected, and the detection accuracy is more likely to be improved, for example, compared to a case where the scanning direction is a vertical direction.
[0088] In the description above, iris recognition is performed on one image (first frame). In a case where the same capture is performed on an image from a second frame, the process can be modified by step S17, which is described in Fig. The process shown in Figure 2 can be aborted. The reason for this is as follows: Since a coordinate of an iris position captured in the first frame has already been determined, a coordinate of an iris position in the second frame can be calculated based on the coordinate of the position in the first frame. That is, in order to capture the iris in the second frame, the scanning circuits can be 93A The data can be scattered around the center position (Pp, Py) of the iris captured in the first frame. Similarly, in a third or subsequent frame, the scanning circuits can be... 93A The data is scattered around a central position of the iris captured in the previous frame. This reduces the amount of data processing required to recognize the iris. <Prozess der zweiten Ausführungsform>
[0089] Fig. Figure 12 illustrates an example of a pattern shape of a particle filter according to a second embodiment. Figures 13A and 13B are schematic diagrams showing specific examples of processing data in a case where the pattern shape of the in Fig. The particle filter shown in section 12 is used.
[0090] In the second embodiment, a “sampling process” shown in Figures 13A and 13B is carried out using a semicircular filter pattern FP, which is shown in Fig. Figure 12 illustrates this. That is, the same process as in the first embodiment is carried out, except that the shape of the filter pattern changes. Since Fig. 13A the same as Fig. Section 6, except that edge image D73 is shown instead of edge image D92, will not have its detailed description repeated. Furthermore, Fig. 13B the same as Fig. In section 7, except that an edge image D73 is shown instead of the edge image D53, the detailed description of it is not repeated.
[0091] The in Fig. 12. The filter pattern FP shown is a semicircular arc-shaped pattern that swells downwards and is projected onto the edge image D73, for example, in the two-dimensional plane, as shown in Fig. 13B is shown.
[0092] The FP filter pattern includes the multiple sampling points. 101 , which are arranged at regular intervals around the circumference, as in the case of the in Fig. 10 filter patterns FP0 are shown. The sampling points 101 The sampled points are divided into points in a left area FPb and points in a right area FPc, with a center point FPa of the filter pattern FP as the boundary. That is, the sampled points 101 , which belong to the left-hand area FPb, can be described as those in Fig. The 10 left group points PGL shown are considered, and the sampling points belonging to the right area FPc are considered. 101 can be considered the in Fig. The 10 right-hand group points PGR shown are to be considered.
[0093] In a case where several particles are scattered around the iris position obtained in the coarse search, the scanning circuits 93B arranged such that the center of the semicircular shape is positioned at a location where different random number components (Ppd, Pyd) correspond to the center position (Pp, Py) of the iris region 92 to be added. At this point, the sampling circuits are 93B arranged in a state in which the bottom of the arc points downwards as the most swelling part of the filter pattern shape in the 3D eyeball model 30.
[0094] The semicircular filter pattern FP is known as the sampling ellipse. 94B on the edge image D73 on the in Fig. The two-dimensional plane shown in Figure 13B is projected. A feature of the shape can be efficiently evaluated in relation to an outline section of a lower half of the iris.
[0095] In general, boundaries between the iris and the white part of the eye tend to appear on the right, left, and lower sides of the iris, while the upper side of the iris is likely a boundary between the iris and an upper eyelid. Therefore, by adjusting the semicircular filter pattern FP, which is in Fig. As shown in Figure 12, an attempt is made to match a scanning curve to each of the right, left, and lower sides of the iris, where the iris boundaries can be accurately captured. In other words, the upper side of the iris, where the accuracy of capturing the iris boundaries deteriorates, can be excluded from the matching targets. Therefore, the iris can be accurately captured.
[0096] The shape of the filter pattern FP is not limited to a semicircular shape, and various arc shapes can be used. The shape of the filter pattern FP can be a C-shape in which part of a circle is incomplete. At this point, it is preferred that the scanning circles are arranged in a state where the bottom, as the most swelling part of the arc in the filter pattern shape in the 3D eyeball model 30, points downwards. <Spezielles Beispiel für den Suchvorgang einer Pupille aus einem Bild>
[0097] The following is a description of the process of locating the pupil from the image. In the "specific example of the process of locating the iris from the image" described above, step S42 of Fig. 4. The black circular image D4t, with a shape similar to the shape of the iris, is used as a template, and this template is scanned onto the image of data D42 while the position P41 of the iris with the most similar feature is sought, thereby identifying the position and radius or diameter of the iris. In the process of searching for the pupil from the image, the pupil is essentially located by a process equivalent to the one described in the “specific example of the process of searching for the iris from the image”.This means that a black circular image with a shape similar to the pupil (the black circle similar to the pupil is smaller than the black circle similar to the iris) is used as a template, and this template is scanned onto the image of data D42 while searching for the position of the pupil with the most similar feature, thus identifying the position and radius or diameter of the pupil. In this way, it is possible to select the object to be searched for from the iris or the pupil by changing the shape of the black circle drawn in the template image. <Prozess der dritten Ausführungsform>
[0098] Fig. Figure 15 illustrates an example of a pattern shape of a particle filter according to a third embodiment. Figures 16A and 16B are schematic diagrams showing specific examples of processing data in a case where the pattern shape of the particle filter is shown in Figure 16A. Fig. The particle filter shown in section 15 is used.
[0099] In the third embodiment, a “scanning process” shown in Figures 16A and 16B is used, employing the method described in Fig. The semicircular filter pattern FP2 shown in Figure 15 is used. That is, the same process as in the first embodiment is carried out, except that the shape of the filter pattern changes. Since Fig. 16A the same as Fig. In section 6, except that edge image D73 is shown instead of edge image D92, the detailed description of it is not repeated. Furthermore, Fig. 16B the same as Fig. 7 with the exception that an edge image D73 is shown instead of the edge image D53, the detailed description of which is not repeated.
[0100] The in Fig. 15. The filter pattern shown, FP2, is a circular pattern in which three circles C1, C2, and C3 with different radii have the same center point and are projected onto the edge image D73, for example, onto the two-dimensional plane, as in Fig. 16B shown.
[0101] The FP2 filter pattern has a large number of sampling points. 201 , 202 and 203 , which are arranged at regular intervals on the circumferences of circles C1, C2 and C3. Furthermore, the sampling points are 201 , 202 and 203 , which each correspond to circles C1, C2 and C3, are set so that they have the same number of scanning points and are arranged radially to be aligned in a radial direction.
[0102] In a case where several particles are scattered around the position of a pupil or iris obtained during the coarse search, scanning circuits are 93C arranged such that the common center point for the three circles is located at a position where different random number components (Ppd, Pyd) correspond to the central position (Pp, Py) of a pupil or iris region. 92 to be added.
[0103] The filter pattern FP2, formed by these three circles, is called a sampling ellipse. 94C projected onto the edge image D73 in the two-dimensional plane, as in Fig. 16B shown.
[0104] Generally, when creating an image obtained by capturing the iris or pupil, the dimensions (the number of pixels in the vertical and horizontal directions) of the iris or pupil change in the image itself, even if the same iris or pupil is captured as a single object. Not only does the dimension of the iris or pupil in the captured image change due to variations in pupil diameter corresponding to the amount of light, but it also changes when the head moves towards or away from the camera, even if the diameter of the iris or pupil remains unchanged. As described above, it is also important to keep the iris or pupil in focus while managing changes in iris or pupil diameter.
[0105] In this respect, it is possible to accommodate changes in iris or pupil diameter using any of the triple circles in the FP2 filter pattern of the third embodiment. That is, even if an iris or pupil captured in an image frame at a specific time t1 changes in diameter in an image frame at a subsequent time t2, circles C1, C2, and C3, being larger or smaller than the circles C1, C2, C3, and C3 that captured the iris or pupil at time t1, can still capture the iris or pupil at time t2. As described above, according to the third embodiment, it is possible to accurately locate the iris or pupil while accommodating changes in iris or pupil diameter.
[0106] It should be noted that the third embodiment of the present invention can be applied to the filter pattern described in the first embodiment or the second embodiment, and can also be applied to the filter pattern described with reference to Fig. 6 was described.
[0107] The image processing device according to the third embodiment of the present invention is briefly described as follows.
[0108] Image processing device with a data processing unit that processes data of a facial image captured to encompass a face, wherein the data processing unit: an edge image is created that stores information regarding a gradient size by filtering the face image to capture an edge in a scanning direction; in a state where a sampling curve is projected onto the edge image, a gradient quantity is extracted as a sample value from each of the positions in the edge image, each corresponding to a plurality of points forming the sampling curve, wherein the sampling curve is virtually positioned on a plane of a three-dimensional model and has a plurality of circles of different radii arranged with the same position as a center point, wherein the extracted gradient quantity as the sample value is a probability assessment target; and a scanning curve with a maximum probability as a pupil or an iris captured among a multitude of scanning curves. <Anwendung der Technik der Bildverarbeitungsvorrichtung>
[0109] The image processing device according to the invention can be used not only for capturing an eye, but also for other applications. Specific examples include head tracking, eyelid detection, and mouth detection. In a case where the image processing device according to the invention is actually applied to head tracking, the dispersion shape size of the particle filter can be adjusted to a head size in an image. Additionally, in a case where the image processing device according to the invention is actually applied to eyelid or mouth detection, the Sobel filter can be used in the longitudinal direction, and the signs of the scanning feature can be divided in the longitudinal direction.
[0110] For example, the respective operations shown in Figs. 2, 4, 5 and 9 can be implemented by hardware such as a dedicated logic circuit, or they can be implemented by software containing a program or data that can be read and executed by different computers for control. <Verbesserung der Genauigkeit>
[0111] Fig. Figure 14 shows an example of a relationship between the application of the invention and a detection error. That is, Fig. Figure 14 illustrates a difference in the detection error between a method to which the invention is applied and a prior art method under conditions where the accuracy deteriorates due to ambient light.
[0112] In Fig. 14 “After Improvement” corresponds to the measurement results of the process to which the invention is applied, and “Before Improvement” corresponds to the measurement results of the process to which the invention is not applied. With reference to Fig. 14 represents an average error of 1.6 pixels per true value in the related technique. On the other hand, the average error of 0.5 pixels per true value is improved in the method according to the invention.
[0113] Here, the characteristics of the embodiments of the image processing device, the image processing method and the image processing program according to the invention are summarized briefly in the following [1] to [6].
[0114] [1] An image processing device comprises: a data processing unit (processing unit) 13 ), which processes the data of a facial image that is captured to encompass a face, where the data processing unit is: an edge image (D73) is generated that stores information regarding a gradient size and whether the gradient is positive or negative, by filtering the face image to capture an edge in a scanning direction; in a state (scanning ellipse) 94A , 94B ), in which a sampling curve (sampling circles 93A, 93B) including an arc-shaped curve, which is virtually positioned on a plane of a three-dimensional model (3D eyeball model 30), is projected onto the edge image, the gradient magnitude and whether the gradient is positive or negative is extracted as a sample value from each of the positions in the edge image, which is a plurality of points (sampling points) 101 ) corresponds to those that form the sampling curve; a probability with respect to a sampling curve is calculated (S103, S104) by placing (S105, S106) points with a positive gradient as probability evaluation targets in a first point group (right group points PGR) positioned on a front side in the sampling direction, below the plurality of points forming the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group (left group points PGL) positioned on a back side in the sampling direction below the plurality of points forming the sampling curve; and a scanning curve with a maximum probability as one pupil or one iris among a multitude of scanning curves (S18f).
[0115] [2] Image processing device according to 1, where the scanning direction in which the data processing unit detects an edge by filtering is a direction in which eyes are oriented in the face image (see Figs. 8A and 8B).
[0116] [3] Image processing device according to 2, where the sampling curve (filter pattern FP) has an arc shape and is projected onto the edge image in a state where a floor is directed downwards as the most swelling part of the arc in the three-dimensional model (See Fig. 12).
[0117] [4] Image processing device according to 1, wherein the data processing unit calculates a probability with respect to the sampling curve by adding (S105, S106) gradient sizes of the points with a positive gradient in the first point group and by adding (S103, S104) gradient sizes of the points with a negative gradient in the second point group.
[0118] [5] Image processing method using an image processing device with a data processing unit that processes data of a face image that is captured to encompass a face, the image processing method includes: a step (S43) to generate an edge image (D73) that stores information regarding a gradient size and whether the gradient is positive or negative, using a filter that detects an edge in the face image in a scanning direction; one step (S103, S105) to extract, in one state (sample ellipse) 94A , 94B), in which a sampling curve (sampling circuit) 93A , 93B ) with an arc-shaped curve, virtually positioned on a plane of a three-dimensional model, projected onto the edge image, the gradient size and whether the gradient is positive or negative, as a sample value from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve; a step (S104, S106) to calculate a probability with respect to a sampling curve, by setting (S105) points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the plurality of points forming the sampling curve, and by setting (S103) points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the plurality of points forming the sampling curve; and a step (S18f) to capture a scanning curve with a maximum probability as a pupil or an iris among a multitude of scanning curves.
[0119] [6] Non-transitory computer-readable medium that stores a program which causes a computer to execute a process for processing an image captured to encompass a face, the procedure includes: a step (S43) to generate an edge image (D73) that stores information regarding a gradient size and whether the gradient is positive or negative, using a filter that detects an edge in the face image in a scanning direction; one step (S103, S105) to extract, in one state (sample ellipse) 94A , 94B), in which a sampling curve (sampling circuit) 93A , 93B ) with an arc-shaped curve, virtually positioned on a plane of a three-dimensional model, projected onto the edge image, the gradient size and whether the gradient is positive or negative, as a sample value from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve; a step (S104, S106) to calculate a probability with respect to a sampling curve, by setting (S105) points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the plurality of points forming the sampling curve, and by setting (S103) points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the plurality of points forming the sampling curve; and a step (S18f) to capture a scanning curve with a maximum probability as a pupil or an iris among a multitude of scanning curves.
[0120] [7] An image processing program that enables a computer to perform the respective steps of the image processing procedure according to [5]. QUOTES INCLUDED IN THE DESCRIPTION
[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature
[0000] JP 2012190351 A [0002, 0003, 0004, 0006, 0008] Cited non-patent literature
[0000] The “Viola-Jones method” is disclosed, for example, in the following document: “P. Viola and MJ Jones, “Rapid Object Detection Using a Boosted Cascade of Simple Features”, IEEE CVPR (2001)
[0035] < / prozessfluss>
Claims
[1] Image processing device comprising: a data processing unit that processes data from a facial image captured to encompass a face, where the data processing unit is: generates an edge image that stores information regarding a gradient size and whether the gradient is positive or negative, by filtering the face image to capture an edge in a scanning direction; in a state in which a sampling curve including an arc-shaped curve virtually positioned on a plane of a three-dimensional model is projected onto the edge image, a sample value is extracted as the information regarding a gradient size and whether the gradient is positive or negative, from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve; a probability with respect to a sampling curve calculated by placing points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the plurality of points forming the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the plurality of points forming the sampling curve; and a sampling curve with a maximum probability as one pupil or an iris is captured under a multitude of scanning curves. [2] Image processing device according to claim 1, wherein the scanning direction in which the data processing unit detects an edge by filtering is a direction in which eyes are oriented in the face image. [3] Image processing device according to claim 2, wherein the scanning curve has an arc shape and is projected onto the edge image in a state in which a bottom is directed downwards as the most swelling part of the arc in the three-dimensional model. [4] Image processing device according to claim 1, wherein the data processing unit calculates a probability with respect to the sampling curve by adding gradient sizes of the points with a positive gradient in the first point group and by adding gradient sizes of the points with a negative gradient in the second point group. [5] Image processing method using an image processing device with a data processing unit that processes data of a face image taken to encompass a face, wherein the image processing method comprises: Generating an edge image that stores information regarding a gradient size and whether the gradient is positive or negative, using a filter that detects an edge in the face image in one scanning direction; Extracting, in a state in which a sampling curve including an arc-shaped curve virtually positioned on a plane of a three-dimensional model is projected onto the edge image, a sample value as the information regarding a gradient size and whether the gradient is positive or negative, from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve; Calculating a probability with respect to a sampling curve by placing points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the multitude of points forming the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the multitude of points forming the sampling curve; and Captured on a scanning curve with a maximum probability as a pupil or an iris among a multitude of scanning curves. [6] Non-transitory computer-readable medium storing a program that causes a computer to execute a process for processing an image captured to encompass a face, the process comprising: Generating an edge image that stores information regarding a gradient size and whether the gradient is positive or negative, using a filter that captures an edge in the face image in one scanning direction; Extracting, in a state in which a sampling curve including an arc-shaped curve virtually positioned on a plane of a three-dimensional model is projected onto the edge image, a sample value as the information regarding a gradient size and whether the gradient is positive or negative, from each of the positions in the edge image corresponding to a plurality of points forming the sampling curve; Calculating a probability with respect to a sampling curve by placing points with a positive gradient as probability evaluation targets in a first point group positioned on a front side in the sampling direction, below the multitude of points forming the sampling curve, and by placing points with a negative gradient as probability evaluation targets in a second point group positioned on a back side in the sampling direction below the multitude of points forming the sampling curve; and Captured on a scanning curve with a maximum probability as a pupil or an iris among a multitude of scanning curves. [7] Image processing program which enables a computer to perform the respective steps of the image processing method according to claim 5.
Citation Information
Patent Citations
device for extracting features from a facial image
DE69333094T2
Eye position detection method and device
US20030118217A1
