Method for testing image suitability for pet muzzle print learning or recognition

Abstract

The present invention provides a method and apparatus capable of determining whether an image is a frontal image in the process of acquiring an image of a nose print for identifying a pet. The method for inspecting the suitability of an image for learning or recognizing a nose print of a pet according to the present invention includes the steps of acquiring an image including a face of the pet, extracting a nose region of the pet from the image, extracting feature points representing an outline of the nose from the nose region, and determining whether the nose region is a frontal image based on a positional relationship of the feature points.

Description

[Technical Field]

[0001] The present invention relates to a method for inspecting the suitability of an image for pet muzzle print learning or recognition, and more particularly to a method and apparatus for determining whether an image of an object that can identify a pet, such as a dog's muzzle print, is suitable for machine learning or recognition based on artificial intelligence and filtering the image. [Background technology]

[0002] In modern society, there is a growing demand for pets, which are companions to humans and provide emotional support. This has led to an increased need for databases of various pet information, including for health management. Managing pets requires pet identification information, similar to human fingerprints, and different objects can be defined for each pet. For example, since each dog has a unique muzzle (the shape of the wrinkles on its nose), each dog's muzzle can be used as identification information.

[0003] As shown in FIG. 1(a), a method for registering a nose print is performed similarly to registering a human fingerprint or face by photographing a pet's face, including its nose (S110), and storing and registering the image including the nose print in a database (server) (S120). Also, as shown in FIG. 1(b), a method for querying a nose print can be performed by photographing a pet's nose print (S130), searching for a nose print and related information that matches the photographed nose print (S140), and outputting the information that matches the photographed nose print (S150). As shown in FIG. 1, the process of registering and querying a pet's nose print allows each pet to be identified and information about the pet to be managed. The pet's nose print information is stored in a database and can be used as data for AI-based learning or recognition.

[0004] However, there are several problems associated with capturing and storing a pet's nose print.

[0005] First, photographs can be difficult to recognize due to factors such as the shooting angle, focus, distance, size, and environment. There have been attempts to apply human facial recognition technology to nose print recognition, but while sufficient data has been accumulated for human faces, there is a lack of data on pet nose prints, resulting in low recognition rates. Specifically, AI-based recognition requires training data that has been processed into a form that the machine can learn from, but there is a lack of data on pet nose prints, making it difficult to recognize nose prints.

[0006] Furthermore, for pet muzzle recognition, an image with clear nose wrinkles is required. However, unlike humans, pets cannot stay still for a while, so it is not easy to obtain a clear image of nose wrinkles. In particular, for muzzle learning or recognition, an image of the front of the nose is preferable, but when a pet animal such as a dog moves, a frontal image may not be captured. Images that are not frontal are not suitable for muzzle learning or recognition, so the images are preferably discarded. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention provides a method and apparatus for determining whether an image is a frontal image in the process of acquiring an image of a nose print for identifying a pet.

[0008] The problems to be solved by the present invention are not limited to those described above, and other problems not described above will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0009] A method for inspecting the suitability of an image for pet nose print learning or recognition according to the present invention includes the steps of acquiring an image including the face of the pet, extracting a nose region of the pet from the image, extracting feature points indicating the outline of the nose from the nose region, and determining whether the nose region is facing forward based on the positional relationship of the feature points.

[0010] According to the present invention, the step of extracting the feature points may include the steps of: applying kernels corresponding to predefined pattern types to the image, respectively, to derive feature vectors representing sensitivities of each pixel of the image to each kernel; and extracting the feature points representing the nose contour from the feature vectors.

[0011] According to the present invention, the step of extracting the feature points may further include a step of comparing a skeleton model generated by connecting the feature points with a nose model of the pet to calibrate the connection relationship and positions of the feature points.

[0012] According to the present invention, the step of determining whether the nose region is in the front direction may include the steps of: constructing a vertical axis connecting feature points representing the upper and lower ends of the nose; and determining whether the face has rotated in the yaw direction by comparing the distance between the feature point representing the left end of the nose and the vertical axis with the distance between the feature point representing the right end of the nose and the vertical axis.

[0013] According to the present invention, the step of determining whether the nose area is facing forward can include the steps of constructing a horizontal axis connecting feature points representing the left and right ends of the nose, and determining whether the face is rotated in the pitch direction based on whether a vector representing the directionality of feature points forming the openings of the pet's nostrils is located within a reference range based on the horizontal axis.

[0014] According to the present invention, the step of determining whether the nose area is in the front direction may include a step of determining whether the face is rotated in the pitch direction based on the position of the feature point corresponding to the uppermost end of the pet's nostrils and the position of the feature point corresponding to the lowermost end of the pet's nostrils relative to the position of the feature point corresponding to the nasal opening.

[0015] According to the present invention, the step of determining whether the nose area is in the front direction may include a step of comparing a left nostril area obtained by connecting feature points constituting the left nostril of the pet with a right nostril area obtained by connecting feature points constituting the right nostril of the pet to determine whether rotation has occurred in the face.

[0016] According to the present invention, the step of determining whether the nose area is in the front direction may include the steps of constructing a vertical axis connecting feature points representing the upper and lower ends of the nose, and comparing the distance between the feature point representing the inner end of the pet's left nostril and the vertical axis with the distance between the feature point representing the inner end of the pet's right nostril and the vertical axis to determine whether rotation has occurred in the face.

[0017] According to the present invention, the step of determining whether the nose area is facing forward can include the steps of constructing a vertical axis connecting feature points representing the upper and lower ends of the nose among the feature points, and determining that the pet's face is not facing forward if the vertical axis overlaps with the pet's nostril area.

[0018] According to the present invention, the method for checking the suitability of an image for pet nose print learning or recognition may further include the steps of: determining that the image including the pet's face is invalid and discarding the image if it is determined that the pet's face is not facing forward; and performing a validity check on the image of the next frame.

[0019] An apparatus for inspecting the suitability of an image for pet nose print learning or recognition according to the present invention includes a camera for photographing the pet and generating an image including the pet's face, and a processor for processing the image to inspect the suitability of the image. The processor may extract a nose region of the pet from the image, extract feature points representing the nose contour from the nose region, and determine whether the nose region is facing forward based on the positional relationship of the feature points.

[0020] According to the present invention, the processor can apply kernels corresponding to predefined pattern types to the image, derive feature vectors representing the sensitivity of each pixel of the image to each kernel, and extract the feature points representing the nose contour from the feature vectors.

[0021] According to the present invention, the processor can compare a skeleton model generated by connecting the feature points with a nose model of the pet to calibrate the connection relationship and positions of the feature points.

[0022] According to the present invention, the processor can construct a vertical axis connecting the feature points representing the upper and lower ends of the nose, and determine whether the face has rotated in the yaw direction by comparing the distance between the feature point representing the left end of the nose and the vertical axis with the distance between the feature point representing the right end of the nose and the vertical axis.

[0023] According to the present invention, the processor forms a horizontal axis connecting the feature points representing the left and right ends of the nose, and can determine whether the face has rotated in the pitch direction based on whether a vector representing the directionality of the feature points forming the openings of the pet's nostrils is located within a reference range based on the horizontal axis.

[0024] According to the present invention, the processor can determine whether the face is rotated in the pitch direction based on the position of the feature point corresponding to the uppermost end of the pet's nostrils and the position of the feature point corresponding to the lowermost end of the pet's nostrils relative to the position of the feature point corresponding to the nasal opening.

[0025] According to the present invention, the processor can compare the left nostril area obtained by connecting the feature points that make up the pet's left nostril with the right nostril area obtained by connecting the feature points that make up the pet's right nostril to determine whether rotation has occurred in the face.

[0026] According to the present invention, the processor can construct a vertical axis connecting the feature points representing the upper and lower ends of the nose, and compare the distance between the feature point representing the inner end of the pet's left nostril and the vertical axis with the distance between the feature point representing the inner end of the pet's right nostril and the vertical axis to determine whether rotation has occurred in the face.

[0027] According to the present invention, the processor constructs a vertical axis connecting the feature points representing the upper and lower ends of the nose among the feature points, and if the vertical axis overlaps with the pet's nostril area, it can determine that the pet's face is not facing forward.

[0028] According to the present invention, if the processor determines that the pet's face is not facing forward, it determines that the image containing the pet's face is invalid, discards the image, and performs a validity check on the image of the next frame. [Effects of the Invention]

[0029] According to the present invention, by extracting feature points that represent the nose contour and determining whether or not rotation has occurred on the face based on the positional relationship of the feature points, it is possible to filter out muzzle print images that are not from the front.

[0030] The effects of the present invention are not limited to those described above, and other effects not described above will be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawings]

[0031] [Figure 1] A general procedure for pet management is provided. [Figure 2] 10 shows a procedure for managing a pet's muzzle print to which the suitability determination of an object image according to the present invention is applied. [Figure 3] 1 is a flowchart of a method for testing the suitability of an image for pet muzzle print learning or recognition. [Figure 4] 10 shows an example of a facial rotation direction. [Figure 5] 1 shows an example of feature points representing the contour of the nose in a muzzle print image according to the present invention. [Figure 6] We present examples of kernels for extracting Haar-like features from muzzle print images and their application. [Figure 7] We present examples of kernels for extracting Haar-like features from muzzle print images and their application. [Figure 8] An example of a process for inspecting and correcting the positions of feature points by comparing a skeleton model with a nose model will be described. [Figure 9] An example of determining whether or not rotation has occurred and the degree of rotation using position information of feature points that represent the nose contour will be described. [Figure 10] An example of determining whether or not rotation has occurred and the degree of rotation using position information of feature points that represent the nose contour will be described. [Figure 11] An example of determining whether or not rotation has occurred and the degree of rotation using position information of feature points that represent the nose contour will be described. [Figure 12] An example of determining whether or not rotation has occurred and the degree of rotation using position information of feature points that represent the nose contour will be described. [Figure 13] An example of determining whether or not rotation has occurred and the degree of rotation using position information of feature points that represent the nose contour will be described. [Figure 14]An example of determining whether or not rotation has occurred and the degree of rotation using position information of feature points that represent the nose contour will be described. [Figure 15] FIG. 1 is a block diagram of an apparatus for testing the suitability of images for pet muzzle print learning or recognition. DETAILED DESCRIPTION OF THE INVENTION

[0032] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily understand and practice the present invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein.

[0033] In order to clearly describe the present invention, parts that are not relevant to the description will be omitted and the same reference numerals will be used throughout the specification to refer to the same or similar components.

[0034] Furthermore, in some embodiments, components having the same configuration will be described only in the representative embodiment using the same reference numerals, and in other embodiments, only configurations that differ from the representative embodiment will be described.

[0035] Throughout the specification, when a part is said to be "connected (or coupled)" to another part, this includes not only "directly connected (or coupled)" but also "indirectly connected (or coupled)" via another member. Furthermore, when a part is said to "comprise" a certain component, this does not mean that it can further include other components, but does not exclude other components, unless otherwise specified.

[0036] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with the contextual meaning of the relevant art, and should not be interpreted in an idealized or overly formal sense unless expressly defined in this application.

[0037] This document mainly describes how to extract identification information by utilizing the shape of wrinkles on a dog's nose (muzzle print), but in this invention, the range of pets is not limited to dogs, and the features used as identification information are not limited to muzzles, but various physical features of pets can be used.

[0038] As described above, there are not enough pet nose print images suitable for AI-based learning or identification, and the quality of pet nose print images is likely to be low. Therefore, nose print images must be selectively stored in a database for AI-based learning or identification. In particular, the present invention provides a filtering method so that only frontal nose print images can be used for learning.

[0039] 2 shows a procedure for managing pet's nose prints to which the suitability determination of an object image according to the present invention is applied. After photographing a pet's nose print, the present invention first determines whether the photographed nose print image is suitable as data for learning or identification. If it is determined to be suitable, it is transmitted to and stored in a recognition server and used as data for subsequent learning or identification.

[0040] The muzzle print management procedure according to the present invention includes a muzzle print acquisition procedure and a muzzle print recognition procedure.

[0041] According to the present invention, when registering a new pet's nose print, an image including the pet is taken, and then a nose print image is extracted from the pet's facial area, and in particular, it is first determined whether the nose print image is suitable for identifying or learning the pet. If the taken image is determined to be suitable for identification or learning, the image is transmitted to a server (artificial intelligence neural network) and stored in a database.

[0042] Similarly, when searching for identification information of a pet through a nose print, an image including the pet is photographed, and then a nose print image is extracted from the face area of ​​the pet, and it is first determined whether the nose print image is suitable for identifying or learning the pet. If the photographed image is determined to be suitable for identification or learning, the image is transmitted to a server, and the identification information of the pet is extracted by matching it with previously stored nose print images.

[0043] In the case of the nose print registration procedure, as shown in FIG. 2, a pet is photographed (S205), and the nose area is first detected from the photographed image of the pet (S210). The photographed image is subjected to a quality inspection to determine whether it is suitable for learning or identification, and a suitable nose print image is output (S215). The output image is then transmitted to a server that constitutes an artificial neural network, where it is stored and registered (S220).

[0044] FIG. 3 is a flow chart of a method for testing the suitability of an image for pet muzzle print learning or recognition.

[0045] The method for testing the suitability of an image for pet nose print learning or recognition according to the present invention includes the steps of acquiring an image including the pet's face (S310), extracting the pet's nose region from the image (S320), extracting feature points representing the nose contour from the nose region (S330), and determining whether the nose region is facing forward based on the positional relationship of the feature points (S340).

[0046] According to the present invention, feature points representing the nose contour are extracted, and based on the positional relationship of the feature points, it is possible to filter out muzzle print images that are not frontal, by determining whether or not a rotation has occurred on the face. If it is determined that the muzzle print image is not a frontal image, the image of that frame is not used for learning or identification. The quality inspection of the muzzle print image according to the present invention is performed independently for each frame in a certain unit, and when a muzzle print image with suitable quality is detected, the image is transmitted to a neural network server and used for learning or identifying the pet.

[0047] FIG. 4 shows an example of the rotation direction of a face. As shown in FIG. 4, the rotation of a three-dimensional object can be expressed in the pitch, roll, and yaw directions. Whether the nose of a three-dimensional pet is facing forward can be determined by determining whether the amount of rotation of the nose in the pitch and yaw directions is within an acceptable range, as shown in FIG. 4. In the case of roll rotation, if roll rotation is detected, a frontal image of the muzzle print can be obtained by rotating the image in the opposite direction to the roll rotation. Therefore, this document will focus on methods for detecting pitch and yaw rotation. According to the present invention, to determine the rotation of the nose in a captured two-dimensional image, the direction and amount of rotation of the nose can be determined by using the characteristic that the pet's nose is symmetrical.

[0048] Therefore, the present invention first defines the feature points that form the nose contour, and then determines the rotation, direction, and amount of rotation of the nose from the relative positions of the feature points, thereby determining whether the muzzle print image is a frontal image. The feature points that form the nose contour may include the upper, lower, left, and right ends of the nose, as well as the upper, lower, left, and right ends of the nostrils.

[0049] For example, as shown in FIG. 5, 34 feature points can be defined for a nose print image, and the numbers of each feature point can be set as follows: -Top of nose: 1, bottom of nose: 18, left of nose: 4, right of nose: 24 - Upper end of left nostril: 8, lower end of left nostril: 12, inner end of left nostril: 10, opening of left nostril: 5-6, 13-14 - Upper edge of right nostril: 28, lower edge of right nostril: 32, inner edge of right nostril: 30, opening of right nostril: 25-26, 33-34

[0050] In addition to the feature point numbers, feature points can be set symmetrically around the vertical center line of the nose, as shown in Figure 5. The direction in which the nose faces (for example, left or right) can be determined based on the positional relationship of these feature points.

[0051] Before explaining the method for determining whether the face has been rotated based on the positional relationship of each feature point, the process for extracting these feature points from a muzzle print image will be described.

[0052] According to the present invention, the step of extracting feature points (S330) includes the steps of applying kernels corresponding to predefined pattern types to the image, respectively, applying the kernels to derive feature vectors representing the sensitivity of each pixel of the image to each kernel, and extracting feature points representing the nose contour from the feature vectors.

[0053] Methods for extracting feature points from a pet's nose print image include Haar-like features, Histogram-of-Gradient (HoG) feature extraction techniques, etc. After extracting local features using Haar-like features or HoG features, feature points can be extracted using principal component analysis or various machine learning techniques.

[0054] For example, Haar-like features are a feature extraction method specialized for extracting boundaries and straight lines. As shown in Figure 6, various shapes of Haar-like kernels can be defined. For example, a Haar-like kernel can be defined as shown in Figure 6 as a kernel for extracting edge features. In Figure 6, (a), (b), (c), and (d) are kernels that are sensitive to edges located within the region, (e), (f), (g), (h), (i), (j), (k), and (l) are kernels that are sensitive to straight lines located within the region, and (m) and (n) are kernels that are sensitive to individuals located at the center of the region. For example, (a) is a kernel that is sensitive to vertical boundaries, (b) is a kernel that is sensitive to horizontal boundaries, and (e) is a kernel that is sensitive to thin straight lines.

[0055] Figure 6 shows 14 Haar-like kernels as an example, but an infinite number of kernels can be defined depending on their size, shape, direction, etc. By performing a convolution operation on the input image with each kernel corresponding to a predefined pattern type, feature vectors can be extracted for each pixel in the image. These feature vectors indicate how sensitive each pixel location in the image was to each Haar-like kernel.

[0056] For example, referring to FIG. 7, the feature point representing the top of the nose often has a horizontal line-shaped boundary component. This indicates a high correlation with kernel (a) in FIG. 6, resulting in a large convolution value. In contrast, the feature point representing the left side of the nose often has a vertical line-shaped boundary component. This indicates a low correlation with kernel (a) in FIG. 6, resulting in a small convolution value.

[0057] Based on this principle, feature information is extracted for each pixel of the image using n kernels while changing the size of the image. In this way, feature information can be extracted for each feature point, and the feature information obtained from a large number of images can be learned using principal component analysis, machine learning methods, or neural network methods.

[0058] While Haar-like features are extracted based on boundary information, HoG feature extraction extracts feature information for each fragment by dividing the image into fragments of a certain size, determining directional information within each fragment, and converting this into a number. Of course, feature information can be extracted using other image processing methods.

[0059] In addition to the above-described Haar-like features and HoG features, the present invention can also use various other feature extraction methods in combination. Feature information extracted from each pixel can be integrated and defined as a feature vector. A model of a muzzle print image can be trained from the extracted feature vector using principal component analysis or various machine learning methods. To find feature points in a newly input image, the feature extractor described above is used to extract feature vectors for each pixel, and the learning model described above is used to select the most likely pixel, thereby finding the feature points.

[0060] Using this method, each feature point can be found independently. However, the multiple feature points defined above have positional relationships. For example, the topmost feature point (number 1) must be located at the highest position on the vertical axis compared to the other feature points, and the feature points representing the nostrils must be located more inward than the feature points representing the outer contour of the nose. To utilize such positional relationships, we can propose a detection method that uses the relative positional relationships between feature points.

[0061] According to the present invention, the step of extracting feature points (S330) may further include a step of comparing a skeleton model generated by connecting the feature points with a pet nose model to calibrate the connection relationship and positions of the feature points.

[0062] For example, as shown on the left side of Figure 8, a straight line is formed by sequentially connecting each feature point, and a skeleton model can be generated using this. This skeleton model can be assumed to be located on the nose contour, and the model can be fitted to the shape of the nose by moving each feature point in the normal direction of the skeleton model. For this process to be successful, the initial position of each feature point must be set, which can be defined using the individual feature point detection method described above.

[0063] Furthermore, through this process, it is possible to check whether individual feature points have been correctly detected, for example, whether feature points of the nostril openings are twisted by crossing each other, and if necessary, to correct the positions of erroneously detected feature points by exchanging the positions of the feature points or aligning the feature points with the nose contour. Furthermore, even if some feature points are missing due to a failure in detection, the positions of the missing feature points can be estimated from the positions of adjacent feature points using a skeletal model.

[0064] Through this process, feature point information of the pet's nose can be efficiently extracted, and using the positional relationship and geometric correlation of the feature points, it can be determined whether the nose is facing forward or whether rotation has occurred on the pitch / roll / yaw axes.

[0065] As described above, after extracting feature points representing the nose contour from the muzzle print image, it is possible to determine whether the image of the pet is a frontal image by analyzing the positional relationship of each feature point. Hereinafter, a method for determining whether the face is rotated based on the feature points of the muzzle print image will be described.

[0066] According to the present invention, the step (S340) of determining whether the nose area is facing forward based on the positional relationship of the feature points includes the steps of constructing a vertical axis 1-18 connecting the feature points representing the upper and lower ends of the nose, and comparing the distance between symmetrical feature points 4 and 24 and the vertical axis 1-18 to determine whether the face has rotated in the yaw direction.

[0067] As shown in Figure 9, the line connecting the upper end 1 and the lower end 18 of the entire nose area (vertical axis) and the line connecting the left end 4 and the right end 24 of the entire nose area (horizontal axis) are used as reference lines to measure the relative positions of corresponding points on the left and right sides. This makes it possible to measure the similarity between the left and right sides of the vertical axis, and to predict which way the nose has rotated, and the degree and extent of rotation in the yaw direction based on the vertical axis.

[0068] For example, in Figure 9, the line connecting 1 and 18 and the line connecting 4 and 24 represent the vertical and horizontal axes of the nose, respectively. When photographed from the normal direction, the distance from 4 to the vertical axis and the distance from 24 to the vertical axis are similar due to symmetry. By measuring these distances between pairs of corresponding points and calculating the difference between the distances, the degree of rotation in the yaw direction can be easily predicted.

[0069] Furthermore, according to the present invention, the step (S340) of determining whether rotation has occurred in the face includes the steps of constructing a horizontal axis 4-24 connecting the feature points representing the left end 4 and the right end 24 of the nose, and determining whether rotation has occurred in the pitch direction of the face based on whether the vector representing the direction of the feature points forming the openings of the pet's nostrils is located within a reference range based on the horizontal axis.

[0070] According to the present invention, by grasping the directional information of the nostril openings 6-5-14-13, 26-25-34-33, it is possible to determine whether the nose is pointing upward or downward. As shown in Figure 10, the directionality of the openings can be defined as a vector based on four points on the nostril openings. For example, when the opening vector is within a reference value range based on the horizontal axis, it can be determined that the pitch direction of the nose is forward; when it is higher than the reference value range, it can be determined that the pitch direction is upward; and when it is lower than the reference value range, it can be determined that the pitch direction is downward.

[0071] Furthermore, according to the present invention, the step (S340) of determining whether rotation has occurred in the face can include a step of determining whether rotation has occurred in the pitch direction of the face based on the positions of the feature points 8 and 28 corresponding to the uppermost ends of the pet's nostrils and the positions of the feature points 12 and 32 corresponding to the lowermost ends of the pet's nostrils relative to the positions of the feature points 6-5-14-13, 26-25-34-33 corresponding to the nasal openings.

[0072] According to the present invention, the pitch direction of the nose can be easily predicted by using the feature points of the top and bottom ends of the nostrils (8, 12, 28, and 32, respectively) and the relative height information of the openings. As shown in FIG. 9, when the nose is facing forward, the top ends 8 and 28 of the nostrils are located higher than the openings 6-5-14-13 and 26-25-34-33, and the bottom ends 12 and 32 are located at the same height as the openings 6-5-14-13 and 26-25-34-33. In contrast, when the nose is facing downward, as shown in FIG. 11, the top end and the openings are located at similar heights. Therefore, using this relative height information, it is easy to determine whether the pitch direction is rotating.

[0073] According to the present invention, the step (S340) of determining whether the nose area is facing forward based on the positional relationship of the feature points includes a step of comparing the left nostril area obtained by connecting the feature points that make up the pet's left nostril with the right nostril area obtained by connecting the feature points that make up the pet's right nostril to determine whether rotation has occurred on the face.

[0074] According to this embodiment, the nose region is divided using polygons connecting feature points, and the area of ​​the region is measured, thereby determining the directionality of the nose. In an ideal situation, if the nose is symmetrical, the areas of the divided regions on the left and right can be assumed to be similar, and the difference in size between the left and right divided regions increases as the nose rotates. This method is less affected by pitch rotation, and is therefore useful when rotation occurs simultaneously in the pitch and yaw directions.

[0075] For example, as shown in Figure 12, the yaw direction can be determined using the nostril area ratio, while being less affected by changes in the pitch angle. In Figure 12, the nose is rotated (pitch) upward and then (yaw) to the left. Because the vertical axis (the line connecting points 1 and 18) is off-center from the nose, it can be difficult to determine yaw using a method that uses corresponding point symmetry. However, when the nostril area is checked, the right nostril is clearly measured as larger, confirming that yaw rotation has occurred in the image.

[0076] According to the present invention, the step (S340) of determining whether the nose area is facing forward based on the positional relationship of the feature points includes the steps of constructing a vertical axis connecting the feature points representing the upper and lower ends of the nose, and comparing the distance between the feature point representing the inner end of the pet's left nostril and the vertical axis with the distance between the feature point representing the inner end of the pet's right nostril and the vertical axis to determine whether rotation has occurred on the face.

[0077] According to this embodiment, the yaw directionality can be obtained using the vertical axis (the line connecting nostrils 1 and 18) and the ratio of the shortest distance to the nostrils. In an ideal situation where the nose is facing forward and symmetrical, the distance from the inner end characteristic point 10 of the left nostril to the vertical axis 1-18 and the distance from the inner end characteristic point 30 of the right nostril to the vertical axis in Figure 13 are similar.

[0078] In the case of the left and right ends of the entire nose region, pitch and yaw rotations may occur simultaneously, which may result in inaccurate results. However, empirically, the distance between the vertical axis 1-18 and the inner feature points 10 and 30 of the nostrils is less affected by pitch and more affected by yaw rotation. Therefore, as shown in Figure 13, the degree of yaw rotation can be estimated by measuring the distance from the vertical axis 1-18 to the inner end of the right nostril (number 10 on the left and number 30 on the right). When the nose rotates significantly in the pitch and yaw directions, one nostril is closer to the vertical axis 1-18 than the other nostril, so if the difference in distance to the two inner ends deviates from a certain value, it can be inferred that yaw rotation has occurred.

[0079] According to the present invention, the step of determining whether rotation has occurred in the face (S340) includes the steps of constructing a vertical axis connecting the feature points representing the upper and lower ends of the nose, and determining that the pet's face is not facing forward if the vertical axis overlaps with the pet's nostril area.

[0080] As shown in Figure 14, there are cases where the nose is rotated so much that one nostril is completely invisible. Such photos are difficult to use for nose print recognition as they are and must be classified and processed separately. If the image is rotated so dramatically that one nostril is not visible, feature point detection will fail, or a feature point position that does not occur naturally will be detected. For example, in Figure 14, the longitudinal axis 1-18 of the nose overlaps with the nostril area, and feature points 25-34 corresponding to one nostril cannot be detected. By determining abnormal feature point placement in this way, it can be determined that the nose was photographed in a form that is not symmetrical from the front.

[0081] In addition, the method for checking the suitability of an image for pet nose print learning or recognition according to the present invention further includes a step of determining that the image including the pet's face is invalid and discarding the image if it is determined that the pet's face is not facing forward.

[0082] In addition to the above-mentioned method, various methods can be used to determine whether the nose print image is a frontal image of a pet by using the positional relationship of feature points.

[0083] According to the present invention, it is possible to filter out non-frontal muzzle print images by extracting feature points that represent the nose contour and determining whether or not a face has been rotated based on the positional relationship of the feature points. If it is determined that a muzzle print image is not a frontal photograph, the image of that frame is not used for pet muzzle print learning or identification. The determination of whether a muzzle print is frontal or not according to the present invention can be performed individually for each of a plurality of frame images, and if the test result for a particular frame is a frontal image, that image can be used to learn or recognize the muzzle print.

[0084] The inspection of a muzzle print image described herein can be performed by an electronic device such as a smartphone or tablet. An apparatus and operation for inspecting the suitability of an image for learning or recognizing a pet's muzzle print according to the present invention will now be described. The inspection of the front view (suitability) of a muzzle print image described above can be performed by a device such as a smartphone or tablet on which a program or application for recognizing a pet's muzzle print is installed.

[0085] 15 is a block diagram of an apparatus for inspecting the suitability of an image for learning or recognizing a pet's nose print. The apparatus 1500 for inspecting the suitability of an image for learning or recognizing a pet's nose print according to the present invention includes a camera 1510 that photographs a pet and generates an image including the pet's face, and a processor 1520 that processes the image and inspects the suitability of the image. The processor 1520 extracts the pet's nose region from the image, extracts feature points representing the nose outline from the nose region, and determines whether the nose region is facing forward based on the positional relationship of the feature points.

[0086] The camera 1510 may include an optical module such as a lens and a CCD (Charge-Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) that generates an image signal from input light, and may generate image data through image capture and provide it to the processor 1520.

[0087] Processor 1520 controls each module of device 1500 and performs calculations necessary for image processing. Processor 1520 can be configured with multiple microprocessors (processing circuits) depending on its function. As described above, processor 1520 can detect a nose region as an object for identifying a pet (e.g., a dog) and determine whether the nose region is in a frontal image.

[0088] Additionally, the device 1500 may include a communication module 1530 for communication with external objects, a memory 1540 for data storage, and a display 1550 for screen output.

[0089] The communication module 1530 can transmit or receive data to or from an external entity via a wired / wireless network. In particular, the communication module 1530 can exchange data for artificial intelligence-based processing through communication with a server. The memory 1540 stores image data and information necessary for image processing, and the display 1550 can display an interface (UI) for muzzle print recognition along with the image being captured. Furthermore, the device 1500 can include various configurations as needed.

[0090] According to the present invention, the processor 1520 applies kernels corresponding to predefined pattern types to an image, derives feature vectors representing the sensitivity of each pixel of the image to each kernel, and extracts feature points representing the nose contour from the feature vector. For example, the processor 1520 derives a feature vector by applying a kernel such as that shown in Figure 6 to a nose print image, and extracts feature points representing the nose contour from the feature vector as shown in Figure 5.

[0091] According to the present invention, the processor 1520 can calibrate the connection relationships and positions of the feature points by comparing a skeleton model generated by connecting feature points with a pet nose model. For example, the processor 1520 can calibrate the connection relationships and positions of the feature points by comparing a skeleton model generated as shown in FIG. 8 with a pet nose model.

[0092] According to the present invention, the processor 1520 can determine whether the face has rotated in the yaw direction by constructing a vertical axis connecting the feature points representing the upper and lower ends of the nose and comparing the distance between the feature point representing the left end of the nose and the vertical axis with the distance between the feature point representing the right end of the nose and the vertical axis. For example, as shown in Figure 9, the processor 1520 can determine whether the face has rotated in the yaw direction by constructing a vertical axis 1-18 connecting the feature points representing the upper and lower ends of the nose and comparing the distance between the feature point 4 representing the left end of the nose and the vertical axis 1-18 with the distance between the feature point 24 representing the right end of the nose and the vertical axis 1-18.

[0093] According to the present invention, the processor 1520 can determine whether the face has rotated in the pitch direction based on whether a vector representing the directionality of the feature points constituting the pet's nostril openings is located within a reference range with respect to the horizontal axis, by configuring a horizontal axis 4-24 connecting the feature points constituting the left and right ends of the nose. For example, the processor 1520 can determine whether the face has rotated in the pitch direction based on whether a vector representing the directionality of the feature points constituting the pet's nostril openings is located within a reference range with respect to the horizontal axis 4-24, as shown in FIG.

[0094] According to the present invention, the processor 1520 can determine whether the face has rotated in the pitch direction based on the positions of the feature points corresponding to the uppermost ends of the pet's nostrils and the feature points corresponding to the lowermost ends of the pet's nostrils relative to the positions of the feature points corresponding to the nostril openings. For example, the processor 1520 can determine whether the face has rotated in the pitch direction based on the positions of the feature points 8 and 28 corresponding to the uppermost ends of the pet's nostrils and the positions of the feature points 12 and 32 corresponding to the lowermost ends of the pet's nostrils relative to the positions of the feature points 6-5-14-13, 26-25-34-33 corresponding to the nostril openings, as shown in FIG.

[0095] According to the present invention, the processor 1520 can determine whether or not a rotation has occurred on the face by comparing the left nostril area obtained by connecting the feature points constituting the left nostril of the pet with the right nostril area obtained by connecting the feature points constituting the right nostril of the pet. For example, the processor 1520 can determine whether or not a rotation has occurred on the face by comparing the left nostril area obtained by connecting the feature points 7-8-9-10-11-12-13 constituting the left nostril of the pet with the right nostril area obtained by connecting the feature points 26-27-28-29-30-31-32-33 constituting the right nostril, as shown in FIG. 12.

[0096] According to the present invention, the processor 1520 can determine whether or not a rotation has occurred on the face by constructing a vertical axis connecting the feature points representing the upper and lower ends of the nose and comparing the distance between the feature point representing the inner end of the pet's left nostril and the vertical axis with the distance between the feature point representing the inner end of the pet's right nostril and the vertical axis. For example, as shown in FIG. 13 , the processor 1520 can determine whether or not a rotation has occurred on the face by constructing a vertical axis 1-18 connecting the feature points representing the upper and lower ends of the nose and comparing the distance between the feature point 10 representing the inner end of the pet's left nostril and the vertical axis 1-18 with the distance between the feature point 30 representing the inner end of the pet's right nostril and the vertical axis 1-18.

[0097] According to the present invention, the processor 1520 constructs a vertical axis connecting the feature points representing the upper and lower ends of the nose among the feature points, and if the vertical axis overlaps with the pet's nostril area, it can be determined that the pet's face is not facing forward. For example, as shown in Figure 14, the processor 1520 constructs a vertical axis 1-18 connecting the feature points representing the upper and lower ends of the nose among the feature points, and if the vertical axis 1-18 overlaps with the pet's nostril area 6-7-8-9-10-11-12-13, it can be determined that the pet's face is not facing forward.

[0098] According to the present invention, if the processor 1520 determines that the pet's face is not facing forward, it determines that the image containing the pet's face is not valid, discards the image, and performs a validity check on the image of the next frame.

[0099] The present embodiment and the drawings attached to this specification merely clearly show a part of the technical ideas contained in the present invention, and it is obvious that all modifications and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical ideas contained in the specification and drawings of the present invention are included in the scope of the present invention.

[0100] Therefore, the spirit of the present invention should not be limited to the described embodiments, and all things that are equivalent to or have equivalent variations within the scope of the claims, as well as the scope of the claims below, should be considered to fall within the scope of the spirit of the present invention.

Claims

1. 1. A method for testing the suitability of an image for pet muzzle print learning or recognition, comprising: acquiring an image including the face of the pet; extracting a nose region of the pet from the image; extracting feature points representing the contour of the nose from the nose region; and determining whether the nose area is in the frontal direction based on the positional relationship of the feature points, The step of extracting feature points includes: applying kernels corresponding to predefined pattern types to the image, respectively, to derive a feature vector representing sensitivity of each pixel of the image to each kernel; extracting the feature points representing the nose contour from the feature vector; and comparing a skeleton model generated by connecting the feature points with a nose model of the pet to calibrate the connection relationship and positions of the feature points; The step of calibrating the connectivity and positions of the feature points includes: calibrating the connection relationships of feature points of nostril openings that extend outside the nostrils in the skeletal model by checking whether the connections between the feature points cross each other and twist.

2. The step of determining whether the nose region is in the front direction includes: constructing a vertical axis connecting feature points representing the top and bottom of the nose; The method of claim 1, further comprising a step of comparing a distance between a feature point representing a left end of the nose and the vertical axis with a distance between a feature point representing a right end of the nose and the vertical axis to determine whether the face has rotated in a yaw direction.

3. The step of determining whether the nose region is in the front direction includes: constructing a horizontal axis connecting feature points representing the left and right ends of the nose; The method of claim 1, further comprising: determining whether the face has rotated in the pitch direction based on whether a vector representing the directionality of the feature points constituting the openings of the pet's nostrils is located within a reference range based on the horizontal axis.

4. The step of determining whether the nose region is in the front direction includes:

2. The method of claim 1, further comprising: determining whether the face has rotated in a pitch direction based on the position of a feature point corresponding to the uppermost end of the pet's nostrils and the position of a feature point corresponding to the lowermost end of the pet's nostrils relative to the position of the feature point corresponding to the nasal opening.

5. The step of determining whether the nose region is in the front direction includes:

2. The method according to claim 1, further comprising a step of comparing a left nostril area formed by connecting feature points constituting the left nostril of the pet with a right nostril area formed by connecting feature points constituting the right nostril of the pet to determine whether rotation has occurred in the face.

6. The step of determining whether the nose region is in the front direction includes: constructing a vertical axis connecting feature points representing the top and bottom of the nose; 2. The method of claim 1, further comprising: comparing the distance between the feature point representing the inner end of the pet's left nostril and the vertical axis with the distance between the feature point representing the inner end of the pet's right nostril and the vertical axis to determine whether rotation has occurred in the face.

7. The step of determining whether the nose region is in the front direction includes: constructing a vertical axis connecting the feature points representing the upper and lower ends of the nose; The method of claim 1 , further comprising the step of: determining that the pet's face is not facing forward if the longitudinal axis and the pet's nostril region overlap.

8. If it is determined that the face of the pet is not facing forward, determining that the image including the face of the pet is invalid and discarding the image; 2. The method of claim 1, further comprising the step of: performing a validity check on the image of the next frame.

9. 1. An apparatus for testing the suitability of an image for pet nose print learning or recognition, comprising: a camera that photographs the pet and generates an image including the face of the pet; a processor for processing the image to check the suitability of the image; The processor: extracting a nose region of the pet from the image; extracting feature points representing the contour of the nose from the nose region; determining whether the nose area is in the frontal direction based on the positional relationship of the feature points; The processor: applying kernels corresponding to predefined pattern types to the image, respectively, to derive a feature vector representing sensitivity of each pixel of the image to each kernel; extracting the feature points representing the nose contour from the feature vector; comparing a skeleton model generated by connecting the feature points with a nose model of the pet to calibrate the connection relationship and positions of the feature points; The device calibrates the connection relationships of the feature points of the nostril openings that extend outside the nostrils in the skeletal model by checking whether the connections between the feature points cross each other and twist.

10. The processor: a vertical axis connecting the feature points representing the upper and lower ends of the nose; The device according to claim 9, wherein the distance between the feature point representing the left end of the nose and the vertical axis is compared with the distance between the feature point representing the right end of the nose and the vertical axis to determine whether the face has rotated in the yaw direction.

11. The processor: constructing a horizontal axis connecting the feature points representing the left and right ends of the nose; The device according to claim 9, wherein the pitch direction rotation of the face is determined based on whether a vector representing the directionality of the feature points constituting the openings of the pet's nostrils is located within a reference range based on the horizontal axis.

12. The processor:

10. The device according to claim 9, wherein whether the face has rotated in the pitch direction is determined based on the position of the feature point corresponding to the uppermost end of the pet's nostrils and the position of the feature point corresponding to the lowermost end of the pet's nostrils relative to the position of the feature point corresponding to the nasal opening.

13. The processor:

10. The device according to claim 9, wherein a left nostril area formed by connecting feature points constituting the left nostril of the pet is compared with a right nostril area formed by connecting feature points constituting the right nostril of the pet to determine whether rotation has occurred in the face.

14. The processor: a vertical axis connecting the feature points representing the upper and lower ends of the nose; The device of claim 9, wherein the distance between the feature point representing the inner end of the pet's left nostril and the vertical axis is compared with the distance between the feature point representing the inner end of the pet's right nostril and the vertical axis to determine whether rotation has occurred in the face.

15. The processor: a vertical axis connecting the feature points representing the upper and lower ends of the nose, The device according to claim 9, wherein if the longitudinal axis and the pet's nostril region overlap, it is determined that the pet's face is not facing forward.

16. The processor: The device of claim 9, wherein if it is determined that the pet's face is not facing forward, the image including the pet's face is determined to be invalid, the image is discarded, and a validity check is performed on the image of the next frame.

Images