Pet biometric information acquisition method and electronic device for conditionally detecting human behavior information and pet biometric information

Abstract

The present invention proposes a method for acquiring biometric information of a pet using a camera, wherein behavioral information of a person (pet owner)'s face and body is acquired together with the biometric information of the pet. The method for acquiring pet biometric information of an electronic device that conditionally detects human behavioral information and pet biometric information according to the present invention comprises: a step of capturing an image containing the person and the pet through a camera provided in the electronic device; a step of outputting the image through a display; a step in which a processor extracts human behavioral information and pet biometric information from the image; a step in which the processor determines whether the image is suitable based on human behavioral information and pet biometric information; and a step of storing the behavioral information and biometric information in memory if the image is determined to be suitable.

Description

Technology Field

[0001] The present invention relates to a method and an electronic device for simultaneously acquiring behavioral information, such as facial expressions and gaze direction of a person raising a pet, and biometric information, such as the pet's nose print or facial shape, depending on whether specific conditions are met. Background Technology

[0002] In modern society, there is a rising demand for companion animals that can live alongside humans and provide emotional support. To support various supplementary services, such as life cycle management and health care, there is an increasing need to database and manage information about these animals. This requires individual identification information, and specific objects that can be used can be defined for each type of animal. For instance, similar to human fingerprints, a dog's nose print (the shape of the nasal folds) is known to differ from one individual to another, and it is generally known that once a dog reaches adulthood (at six months of age), these characteristics do not change for the remainder of its life. Therefore, in the case of dogs, nose prints can be utilized as a means of identification.

[0003] However, unlike humans who can assume specific postures or use various sensors, it is not easy for pets to do so. Since verbal communication is impossible, methods such as using physical force or tools like restraint frames to forcibly fix the pet in a specific posture or make contact with sensors could be considered; however, this is not advisable in terms of protecting the pet's rights and minimizing resistance from the owner. Furthermore, since both a person to restrain the pet and a person to operate the biometric data acquisition device are required, there is a practical problem in that it is very difficult to collect biometric data alone.

[0004] As society becomes one where humans and animals coexist, situations requiring the identification of pets and their owners are occurring more frequently. For instance, when a pet is being picked up from a boarding facility (such as a dog hotel) upon entry, the facility has no way to verify if the person is indeed the owner. Furthermore, regarding the issue of abandonment, cases where owners deny ownership of their pets are occurring frequently. The problem to be solved

[0005] The present invention proposes a method for acquiring biometric information of a pet using a camera, wherein behavioral information of a person (pet owner)'s face and body is acquired together with the biometric information of the pet.

[0006] By doing so, the present invention enables the acquisition of images more suitable for identification or learning when capturing images for the biometric recognition of a pet. Furthermore, the present invention facilitates the identification of a pet and provides a solution to pet ownership disputes and denial of ownership that may occur in specific situations.

[0007] The problems solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0008] A method for acquiring pet biometric information of an electronic device that conditionally detects human behavioral information and pet biometric information together according to the present invention comprises: a step of capturing an image including the person and the pet through a camera provided in the electronic device; a step of outputting the image through a display; a step in which a processor extracts human behavioral information and pet biometric information from the image; a step in which the processor determines whether the image is suitable based on the human behavioral information and pet biometric information; and a step of storing the behavioral information and the biometric information in memory if the image is determined to be suitable.

[0009] According to an embodiment of the present invention, the step of capturing the image may include, firstly, capturing an image of the person and the pet with a front camera that captures the image facing the display direction of the display.

[0010] According to an embodiment of the present invention, if the front camera cannot be activated, the method may include the step of capturing the image using a rear camera provided on the opposite side of the front camera instead of the front camera.

[0011] According to an embodiment of the present invention, whether the front camera can be activated is determined based on attribute information of the front camera, and the attribute information may include whether manual focus is supported, shooting format, and shooting size.

[0012] According to an embodiment of the present invention, the step of determining whether the image is suitable may include the step of determining whether the image is suitable based on the facial expression of the person or the direction of the person's gaze.

[0013] According to an embodiment of the present invention, if the facial expression of the person is determined to be a smiling expression or a neutral expression, the image is determined to be suitable, and if the facial expression of the person is determined to be an expression other than a smiling expression or a neutral expression, the image may be determined to be unsuitable.

[0014] According to an embodiment of the present invention, the step of determining the suitability of the image may include the step of evaluating the quality of a feature region containing the bio-information of the pet in the image.

[0015] According to an embodiment of the present invention, the quality may include clarity, brightness, and size.

[0016] According to an embodiment of the present invention, the evaluation conditions for the quality may be set differently depending on whether the camera is a front camera that captures an image facing the display direction of the display or a rear camera provided on the opposite side of the front camera.

[0017] An electronic device for conditionally detecting human behavioral information and pet biometric information together according to the present invention comprises: a camera for capturing an image; a display for visually displaying the image; a processor functionally coupled with the display and the camera; and a memory functionally coupled with the processor. The processor is configured to capture an image of the human and pet through the camera, output the image through the display, extract human behavioral information and pet biometric information from the image, determine the suitability of the image based on the human behavioral information and pet biometric information, and if the image is determined to be suitable, store the behavioral information and biometric information together with the image in the memory. Effects of the invention

[0018] According to the present invention, in the process of acquiring an image for biometric recognition of a companion animal, a quality evaluation of the image reflecting human behavioral information is performed to determine whether to acquire the image, thereby allowing an image more suitable for learning or identification to be acquired.

[0019] Furthermore, human behavioral information and pet biometric information can be acquired simultaneously, ensuring temporal and spatial simultaneity. Through this, a correlation can be established between the human behavioral information and the pet's biometric information, enabling the performance of a complex authentication process. Brief explanation of the drawing

[0020] FIG. 1 illustrates the appearance of an electronic device according to the present invention. FIG. 2 is a block diagram showing the configuration of an electronic device according to the present invention. FIG. 3 illustrates a system including an electronic device for acquiring biometric information of a pet and a server for processing biometric information. Figure 4 is a flowchart illustrating a method for acquiring pet biometric information that conditionally detects human behavioral information and pet biometric information together. Figure 5 is a diagram illustrating the process of acquiring human behavioral information and pet animal biological information. Figures 6 and 7 illustrate examples of screens displayed during the process of acquiring human behavioral information and pet biometric information. Figures 8 and 9 are flowcharts illustrating the process of selecting a camera to capture images for acquiring human behavioral information and pet biometric information. FIG. 10 illustrates an example of an interface for selecting a camera to perform video recording. Figure 11 is a flowchart illustrating the process of examining the compatibility between the behavioral information of a person (HM) and the biometric information of a companion animal. Figure 12 is a flowchart illustrating the process of checking the suitability of human behavior information. FIGS. 13 and 14 illustrate examples of a shooting guide interface based on the results of a suitability test in the process of acquiring human behavioral information and pet biometric information. Specific details for implementing the invention

[0021] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein.

[0022] To clearly explain the present invention, parts unrelated to the explanation have been omitted, and the same reference numerals are used for identical or similar components throughout the specification.

[0023] In addition, in various embodiments, components having the same configuration are described using the same reference numerals only in the representative embodiment, and in other embodiments, only configurations different from the representative embodiment are described.

[0024] Throughout the specification, when a part is described as being "connected (or combined)" with another part, this includes not only cases where they are "directly connected (or combined)" but also cases where they are "indirectly connected (or combined)" with other members in between. Furthermore, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0025] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0026] The following describes a method for acquiring biometric information of a pet (PT) and an electronic device (1) for performing the same, which conditionally detects behavioral information of a person (HM) and biometric information of a pet (PT) together according to an embodiment of the present invention. The biometric information of a pet (PT) is acquired for the identification of the pet (PT). The biometric information of a pet (PT) may include an iris, facial features, body features, and nose wrinkle features (nose prints). Although this document focuses on describing the nose prints of dogs and the facial features of cats as biometric information of pets (PT), the scope of pets (PT) in the present invention is not limited to dogs and cats, and various physical features of pets (PT) may be used as characteristics used as identification information, not limited to nose prints.

[0027] According to the present invention, an electronic device (1) photographs a pet (PT) using a camera (10) and obtains biometric information of the pet (PT) from the captured image. The electronic device (1) may be a mobile device such as a smartphone or tablet. Additionally, the electronic device (1) may correspond to a computing device (e.g., a laptop computer or a desktop computer). In the case of a foldable mobile device such as a foldable phone, a front camera (11) may be present before and after folding, respectively. In the electronic device (1) according to the present invention, the front camera (11) may be provided inside or outside the display (20).

[0028] FIG. 1 illustrates an example of an electronic device (1) in which a front camera (11) is mounted inside a display area. FIG. 1 (a) shows the front (F) of the electronic device (1) in which a front camera (11) is mounted inside a display area, and FIG. 1 (b) shows the rear (R) of the electronic device (1). The exterior of the electronic device (1) may consist of a frame, a display (20), and cameras (10) (front camera (11), rear camera (12)). In FIG. 1, the front camera (11) may be configured inside the screen area of ​​the display (20). The front camera (11) and the rear camera (12) may each consist of multiple cameras. Meanwhile, the front camera (11) may be mounted outside the display area.

[0029] Referring to FIG. 2, the electronic device (1) according to the present invention includes a camera (10), a display (20), a processor (30), and a memory (40). Optionally, the electronic device (1) may further include a gyroscope sensor (61), a touch sensor (62), or an ambient light sensor (63). Optionally, the electronic device (1) may further include a communication module (50), a speaker (71), or a microphone (72).

[0030] The display (20) is a device that outputs visual information. The display (20) may include a panel and a control circuit. The panel may be equipped with a touch sensor (62). In particular, the display (20) may output an image (preview image, captured frame) of the pet (PT) being filmed. Additionally, the display (20) may output an interface that provides information to the user during the process of acquiring the pet's (PT) biometric information.

[0031] A camera (10) is a device for capturing images (still images or video images). The camera may include one or more image sensors, lenses, camera controllers, image processing processors, and flashes. The camera (10) may include a front camera (11) and a rear camera (12). The camera (10) may capture images to acquire biological information of a pet (PT) and behavioral information of a person (HM). The characteristics of the camera (e.g., focus, exposure) may be controlled so that feature objects for acquiring biological information of the pet (PT) can be properly identified. The front camera (11) is a camera that captures images facing the display direction of the display (20) as in FIG. 1, and the rear camera (12) is a camera that captures images facing the opposite direction of the display direction of the display (20) as in FIG. 2. The camera may include not only a camera built into the electronic device (1) but also an external camera coupled to the electronic device (1) via a wired or wireless connection.

[0032] The processor (30) can perform operations and data processing to perform the operation of the electronic device (1), and control each module of the electronic device (1). The processor (30) can execute one or more software programs stored in memory (40). The processor (30) may include a general-purpose processor (e.g., CPU, AP) that runs an operating system managing the overall operation of the electronic device (1). Additionally, the processor (30) may include a dedicated processor (e.g., GPU, NPU) for specific computation processing. The processor (30) may include one or more processing circuits. The processor (30) can perform data processing and control operations to acquire biological information of a pet (PT) and behavioral information of a person (HM).

[0033] The memory (40) can store programs, such as an operating system and applications for the operation of the electronic device (1), and data used during the execution of the programs. The memory (40) may include volatile memory (e.g., DRAM) and non-volatile memory (e.g., flash memory). The memory (40) can store images for biometric recognition of a pet (PT). Additionally, the memory (40) can store biometric information of the pet (PT) and behavioral information of a person (HM) together with the images.

[0034] The communication module (50) provides an environment in which the electronic device (1) can communicate with an external entity (e.g., a server, a base station, an IoT device, a mobile device). The communication module (50) may include an antenna, an RF circuit, and a communication processor. The communication module (50) may provide data received from an external entity to the processor (30). The communication module (50) may convert the data provided by the processor (30) and transmit it to the external entity. The communication module (50) may receive video data for biometric authentication of a pet (PT) provided by the processor (30) and transmit the video data to the server (2). As illustrated in FIG. 3, the electronic device (1) may communicate with the server (2). The electronic device (1) may acquire biometric information of the pet (PT) and transmit the acquired biometric information to the server (2). The server (2) may store the biometric information in a database. The server (2) may perform data processing for learning or identification of the pet (PT) from the biometric information. The server (2) can generate identification information of a pet (PT) from biometric information and behavioral information of a person (HM) and transmit the identification information of the pet (PT) to an electronic device (1).

[0035] An electronic device (1) for acquiring biometric information of a pet (PT) according to the present invention comprises a camera (10) for capturing images, a display (20) for visually displaying images, a processor (30) functionally coupled with the display (20) and the camera (10), and a memory (40) functionally coupled with the processor (30).

[0036] The processor (30) is configured to capture images of a person (HM) and a pet (PT) through a camera (10), output images through a display (20), extract behavioral information of the person (HM) and biometric information of the pet (PT) from the images, determine whether the images are suitable based on the behavioral information of the person (HM) and the biometric information of the pet (PT), and if the images are determined to be suitable, store the behavioral information and biometric information together with the images in a memory (40).

[0037] FIG. 4 is a flowchart illustrating a method for acquiring biometric information of a pet (PT) that conditionally detects behavioral information of a person (HM) and biometric information of a pet (PT). The method for acquiring biometric information of a pet (PT) that conditionally detects behavioral information of a person (HM) and biometric information of a pet (PT) according to the present invention includes the steps of: capturing an image containing a person (HM) and a pet (PT) through a camera (10) provided in an electronic device (1) (S410); outputting the image through a display (20) (S420); extracting behavioral information of a person (HM) and biometric information of a pet (PT) from the image (S430); determining whether the image is suitable based on behavioral information of a person (HM) and biometric information of a pet (PT) (S440); and if the image is determined to be suitable, storing the behavioral information and biometric information in a memory (40) (S450). The image capture, extraction of human behavioral information and animal biological information, and suitability check according to the present invention may be performed for every frame, or may be repeated at regular frame intervals or regular time intervals.

[0038] According to the present invention, when capturing a video for biometric recognition of a pet (PT), behavioral information of a person (HM) is considered in the process of determining whether the video is suitable for recognition or learning. For example, if the person (HM) has a flustered expression or is not looking straight ahead, it is highly likely that the person is not ready to capture a video with the pet (PT). In a state where the person is not ready to capture, the pet (PT) is also highly likely to not look straight ahead or to move its head, and in this state, even if a video is captured, the likelihood of capturing a video of suitable quality is very low.

[0039] Therefore, in the process of determining whether to acquire an image, behavioral information of a person (HM) is analyzed, and it is determined whether the behavioral information satisfies the conditions for image acquisition. If the behavioral information of the person (HM) satisfies the conditions for image acquisition and other acquisition conditions, the image is determined to be suitable for biometric recognition of a pet, and the image can be stored.

[0040] In addition, a video in which a pet (PT) and a person (HM) are filmed together can be used to identify who the owner of the pet (PT) is. Therefore, when filming a video for the biometric identification of a pet (PT), by ensuring that the animal (PT) and the person (HM) are filmed together, information about the owner can be used as information for identifying the identity of the pet (PT).

[0041] According to the present invention, in the process of acquiring an image for biometric recognition of a pet (PT), when determining whether the image is suitable for the identification of the pet (PT), the suitability of the feature objects (e.g., face, nose, eyes, etc.) for the biometric recognition of the pet (PT) is considered together with the behavioral information of the person (HM) being filmed. When the pet (PT) and the person (HM) are filmed together, a suitability test for the feature objects of the pet (PT) and the feature objects of the person (HM) may be performed simultaneously to determine whether the filmed image is suitable for the identification of the pet (PT). The test for the feature objects of the pet (PT) may be performed by testing the clarity, brightness, and size of the image containing the feature objects. The test for the feature objects of the person (HM) may be similar to the test method for the feature objects of the pet (PT). Furthermore, the test for the feature objects of the person (HM) includes a process of confirming that the person (HM) is filming an image for the biometric recognition of the pet (PT) through the person's facial expressions and gaze.

[0042] In step S410, an application for biometric recognition of the pet (PT) is executed, and the camera (10) is activated to begin video recording. As the camera (10) for video recording, a front camera (11) or a rear camera (12) may be used. As shown in FIG. 5, a front camera (11) suitable for recording a person (HM) and a pet (PT) together may be used first. That is, the step of recording video (S410) may include the step of recording a video of a person (HM) and a pet (PT) using a front camera (11) that records a video facing the display direction of the display (20) first.

[0043] The process of acquiring biometric information of a pet using a front camera (11) can be performed by a user (HM) holding an electronic device (1) and photographing the pet (PT) as illustrated in FIG. 5. Compared to using a rear camera (12), there are several advantages to using a front camera (11) to perform photography for acquiring biometric information of the pet (PT). Generally, pets such as dogs do not perceive photography and therefore do not look at the camera but look in a different direction. In addition, most pets cannot stay still and have a habit of constantly moving their heads, making it very difficult to take a frontal photograph suitable for biometric recognition. Furthermore, since most users are not skilled at photography, it is difficult for most users to capture an image suitable for biometric recognition of the pet.

[0044] On the other hand, when photographing a pet using the front camera (11) as shown in FIG. 5, the user can view the video of the pet currently being photographed in real time, so a video that is more suitable for biometric recognition of the pet can be acquired. In addition, the user can photograph while holding the pet, and since the user and the pet are photographed in a more comfortable state, a video that is more suitable for biometric recognition can be acquired.

[0045] FIGS. 6 and 7 illustrate examples of screens displayed during the process of acquiring behavioral information of a person (HM) and biometric information of a companion animal (PT). FIG. 6 illustrates a screen in which a person and a dog are photographed together, and FIG. 7 illustrates a screen in which a person and a cat are photographed together. When a person and a dog are photographed together as in FIG. 6, the dog's nose is used as a feature object, and an animal feature area (100) containing the feature object is extracted. Additionally, the person's face is used as a feature object, and a person feature area (200) containing the feature object is extracted. When a person and a cat are photographed together as in FIG. 7, the cat's face is used as a feature object, and an animal feature area (100) containing the feature object is extracted. Additionally, the person's face is used as a feature object, and a person feature area (200) containing the feature object is extracted.

[0046] In order to obtain biological information of a pet, the front camera (11) can photograph the face of the pet. Prior to the shooting of the front camera (11), the processor (30) can determine whether to activate the front camera (11). That is, the processor (30) can determine whether to take a video to obtain biological information of the pet using the front camera (11). If the front camera (11) can be activated, the processor (30) can activate the front camera (11). If the front camera (11) cannot be activated, the processor (30) can activate the rear camera (12) provided on the opposite side of the front camera (11) instead of the front camera (11).

[0047] As illustrated in FIG. 8, the step (S410) of capturing a video including a person (HM) and a pet (PT) includes: a step (S805) of checking attribute information of the front camera (11); a step (S810) of determining whether the front camera (11) can be activated based on the attribute information; a step (S815) of activating the front camera (11) if the front camera (11) can be activated; and a step (S820) of activating the rear camera (12) provided on the opposite side of the front camera (11) in place of the front camera (11) if the front camera (11) cannot be activated.

[0048] According to another embodiment, the user may be allowed to select which camera to use. According to the present invention, the processor (30) checks attribute information of the front camera (11) and determines whether the front camera (11) can be activated based on the attribute information of the front camera. The attribute information of the front camera (11) may include whether manual focus is supported, shooting format, shooting size, and hardware level information. The attribute information of the front camera (11) may be stored as data in a camera management program. For example, the processor (30) may determine that the front camera (11) is a camera that can be used to acquire biological information of a pet if the front camera (11) is a camera that supports manual focus, the shooting format and shooting size satisfy pre-set conditions, and the hardware level of the camera meets a reference level.

[0049] The processor (30) can output attribute information of the front camera (11) and whether the front camera (11) can be activated through the display (20). By displaying attribute information of the front camera (11) and whether the front camera (11) can be activated, the user can check whether the front camera (11) can be used to obtain biometric information of the pet.

[0050] If the front camera cannot be activated, the processor (30) can activate the rear camera (12). That is, the processor (30) can perform a shot for biometric recognition of a pet using the rear camera (12) without activating the front camera (11).

[0051] The processor (30) can output an interface that allows the user to select either the front camera (11) or the rear camera (12) when the front camera (11) is enabled. For example, when the front camera is enabled, the processor (30) can output an interface that allows the user to select the camera. For example, by default, the rear camera (12) is enabled to display a preview image, and an icon (400) that allows switching cameras can be output at the bottom right of the application screen as in FIG. 10. When a touch input to the icon (400) is detected by the touch sensor (62), the processor (30) can enable the front camera (11) and display the video captured by the front camera (11) as a preview image on the display (20). When a touch input to the icon (400) is detected again while the front camera (11) is shooting, the processor (30) can output the video captured by the rear camera (12) as a preview image on the display (20). As another example, a list can be displayed to allow the user to select either the front camera (11) or the rear camera (12) before starting to shoot.

[0052] When the front camera (11) is selected, the processor (30) activates the front camera (11) to acquire an image for biometric recognition of the pet. When the rear camera (12) is selected, the processor (30) activates the rear camera (12) to acquire an image for biometric recognition of the pet.

[0053] As illustrated in FIG. 9, in the step (S410) of capturing a video including a person (HM) and a pet (PT), the method includes a step (S905) of checking attribute information of the front camera (11) and a step (S910) of determining whether the front camera (11) can be activated based on the attribute information of the front camera (11). Additionally, a step (S940) of outputting the attribute information of the front camera (11) and whether it can be activated through a display (20) may be executed. If the front camera (11) can be activated, a step (S915) of outputting an interface to select one of the front camera (11) and the rear camera (12) may be executed. A step (S920) of determining whether the front camera (11) is selected through the interface, a step (S925) of activating the front camera (11) if the front camera (11) is selected, and a step (S930) of activating the rear camera (12) if the front camera (11) is not selected are executed. Subsequently, a process of capturing video for biometric recognition of a pet through the selected camera may be executed.

[0054] When a video is captured, biometric information of the pet (PT) and behavioral information of the person (HM) are extracted from the frames of the captured video. The biometric information of the pet (PT) can be extracted from the feature region (100) of the pet (PT), and the behavioral information of the person (HM) can be extracted from the feature region (200) of the person (HM). As a condition for extracting the biometric information of the pet (PT), a judgment of suitability regarding the quality of the feature region (100) and the behavioral information of the person (HM) can be performed.

[0055] According to the present invention, in determining whether a captured image is suitable for animal biometric recognition, a quality inspection of the animal feature region (100) is performed together with an inspection of the human feature region (200). The inspection of the human feature region (200) may include a quality inspection such as clarity, size, and brightness, and an inspection of behavioral information of the person (HM). The behavioral information of the person (HM) includes facial expression information and gaze information.

[0056] FIG. 11 is a flowchart illustrating the process of examining the suitability of behavioral information of a person (HM) and biometric information of a companion animal (PT). As illustrated in FIG. 11, when an image is captured, the step of performing a suitability examination of the behavioral information of the person (HM) (S1110) and the step of performing a suitability examination of the biometric information of the companion animal (PT) (S1120) may be executed.

[0057] The process of performing a suitability check on behavioral information of a person (HM) may include a step of determining whether an image is suitable based on the person's facial expression or gaze direction in the image.

[0058] FIG. 12 is a flowchart illustrating the process of checking the suitability of behavioral information of a person (HM). Referring to FIG. 12, the step of performing a suitability check of behavioral information of a person (HM) (S1110) includes the step of analyzing the facial expression of the person (HM) in an image (S1205) and the step of determining whether the facial expression is a suitable expression (smiling expression or neutral expression) (S1210).

[0059] In the step (S1205) of analyzing the facial expression of a person (HM), a process is executed to extract a feature region (200) from an image that includes an area corresponding to the person (HM) and an object capable of identifying the expression of the person (HM). Expression information (blank expression, smile, embarrassment, anger, sadness) regarding the face region of the person (HM) can be extracted as behavioral information of the person. Various methods, including algorithms such as Eigenfaces, LBP (Local Binary Patterns), CNN (Convolutional Neural Networks), and SVM (Support Vector Machines), may be used to analyze the expression of the person (HM).

[0060] If the facial expression is not an appropriate expression (e.g., an angry expression or a flustered expression), the video of the face is determined to be unsuitable (S1230). For example, as shown in FIG. 13, if a person's face is obscured and no expression is detected, or if a flustered expression is detected, the video is determined to be unsuitable, and a guidance message (320) is output to instruct the person (HM) to make a smiling expression.

[0061] If the facial expression is an appropriate expression, a step of analyzing the gaze direction of the person (HM) (S1215) and a step of determining whether the person's gaze direction is frontal (S1220) are executed. In the step of analyzing the gaze direction of the person (HM) (S1215), the eyes within the person's face region are detected as feature objects, and an algorithm for detecting the gaze direction from the eye image may be executed. For example, the gaze direction can be detected by detecting the position of the pupil within the eye region and comparing the position of the eye with the position of the pupil. An algorithm such as PCCR (Pupil Center Corneal Reflection) may be used to detect the gaze direction.

[0062] If the direction of gaze is determined to be frontal, the captured image is determined to be suitable (S1225). If the direction of gaze is determined not to be frontal, the captured image is determined to be unsuitable (S1230). For example, as shown in FIG. 13, if a person's gaze direction is not frontal but upward, a guidance message (320) may be output to instruct the person (HM) to look at the front of the camera. On the other hand, as shown in FIG. 14, if a person's facial expression is smiling and the direction of gaze is frontal, the image may be determined to be suitable. In this case, a guidance message (300) may be output to instruct the person to continue maintaining the same facial expression and gaze.

[0063] In Fig. 12, an example is shown where facial expression analysis is performed first and then gaze analysis is performed, but gaze analysis may be performed first, or facial expression analysis and gaze analysis may be performed in parallel. Alternatively, only one of facial expression analysis and gaze analysis may be used.

[0064] Meanwhile, after an image of a feature object for biometric recognition of a pet is acquired through a front camera (11) or a rear camera (12), a quality evaluation may be performed to determine whether the image of the feature object is suitable for learning or identification of the pet. According to the present invention, a processor (30) can evaluate the quality of the image of the feature object. Conditions defined for evaluating the quality of the image of the feature object may include clarity, brightness, and size. The quality evaluation conditions may differ depending on whether the camera that captured the image of the feature object is a front camera (11) or a rear camera (12). For example, when determining image quality through the size of the image of the feature object, the size condition of the image captured by the front camera (11) may be set to be larger than that of the rear camera (12). Generally, the front camera (11) is configured as a camera with lower performance than the rear camera (12). For example, the front camera (11) has lower overall image quality compared to the rear camera (12) and may not support the Optical Image Stabilization (OIS) function, which is a hand shake prevention function. Therefore, when an image is captured through the front camera (11), the quality evaluation of the image of the feature object can be performed under higher quality conditions. Thus, even when captured through the front camera (11), the biometric recognition of the pet can be performed smoothly through the image of the feature object.

[0065] The results of the suitability test for the pet (PT)'s biological information in the captured video can be displayed on the screen. If the pet (PT) is identified as a dog in the captured video and the quality evaluation result for the feature area (200) containing the pet (PT)'s feature object is determined to be unsuitable, a message (310) guiding the dog to look at the front of the camera can be displayed as shown in FIG. 13. If the quality evaluation result for the feature area (200) containing the pet (PT)'s feature object is determined to be suitable, a message (300) guiding the dog to continue maintaining the corresponding posture can be displayed as shown in FIG. 14.

[0066] If it is determined that both the behavioral information of the person (HM) and the biometric information of the pet (PT) in the video are suitable, the captured video is stored in memory (40). The video stored in memory (40) can be transmitted to the server (2) through the communication module (50).

[0067] When the processor (30) of the electronic device (1) transmits an image of a feature object to the server (2), it may transmit, along with the image data, related information such as behavior information of a person (HM), identity information, and camera information that captured the image. The server (2) may store and manage the behavior information and identity information of the person (HM) in association with the identity information of the pet (PT). Additionally, the server (2) can determine which camera captured the image through the camera information. The server (2) can determine how to process the feature object through the camera information. That is, the server (2) can process the image differently depending on whether the camera that captured the image is a front camera (11) or a rear camera (12).

[0068] The embodiments and drawings attached to this specification merely clearly illustrate a part of the technical concept included in the present invention, and it is obvious that variations and specific embodiments that can be easily deduced by a person skilled in the art within the scope of the technical concept included in the specification and drawings of the present invention are all included within the scope of the rights of the present invention.

[0069] Accordingly, the scope of the present invention should not be limited to the described embodiments, and all things equivalent to or having equivalent variations to the claims set forth below, as well as the claims themselves, shall be considered to fall within the scope of the concept of the present invention.

Claims

Claim 1 A method for acquiring pet biometric information using an electronic device that conditionally detects human behavioral information and pet biometric information together, comprising: a step of capturing an image containing a person and a pet through a camera provided in the electronic device; a step of outputting the image through a display; a step of extracting human behavioral information and pet biometric information from the image; a step of determining whether the image is suitable based on the human behavioral information and pet biometric information; and a step of storing the behavioral information and pet biometric information in memory if the image is determined to be suitable, wherein the step of determining whether the image is suitable is a step of determining whether the human behavioral information extracted from the image satisfies a pre-set valid condition, wherein the valid condition includes the human’s gaze being directed forward or making a pre-set facial expression; and a step of performing a quality evaluation of a feature region containing pet biometric information only when the human behavioral information satisfies the valid condition, and determining the image as suitable if the result of the quality evaluation satisfies a standard. Claim 2 A method for acquiring biometric information of a pet according to claim 1, wherein the step of capturing the image includes, firstly, capturing an image of the person and the pet with a front camera that captures an image facing the display direction of the display. Claim 3 A method for acquiring biometric information of a pet, comprising the step of capturing the image using a rear camera provided on the opposite side of the front camera in place of the front camera when the activation of the front camera is impossible in paragraph 2. Claim 4 In paragraph 3, the activation capability of the front camera is determined based on attribute information of the front camera, and the attribute information includes whether manual focus is supported, shooting format, and shooting size, a method for acquiring biometric information of a pet. Claim 5 delete Claim 6 A method for acquiring biometric information of a pet according to claim 1, wherein if the facial expression of the person is determined to be a smiling expression or a neutral expression, the image is determined to be suitable, and if the facial expression of the person is determined to be an expression other than a smiling expression or a neutral expression, the image is determined to be unsuitable. Claim 7 A method for acquiring biometric information of a pet according to claim 1, wherein if the direction of the person's gaze is determined to be a frontal direction, the image is determined to be suitable, and if the direction of the person's gaze is determined to be other than a frontal direction, the image is determined to be unsuitable. Claim 8 A method for acquiring biometric information of a pet according to claim 1, wherein the step of determining the suitability of the image comprises the step of evaluating the quality of a feature region containing the biometric information of the pet in the image. Claim 9 In claim 8, the above quality is a method for acquiring pet biometric information including clarity, brightness, and size. Claim 10 A method for acquiring pet biometric information using an electronic device that conditionally detects human behavioral information and pet biometric information together, comprising: a step of capturing an image containing a person and a pet through a camera provided in the electronic device; a step of outputting the image through a display; a step of extracting human behavioral information and pet biometric information from the image; a step of determining whether the image is suitable based on the human behavioral information and pet biometric information; and a step of storing the behavioral information and pet biometric information in memory if the image is determined to be suitable, wherein the step of determining whether the image is suitable includes a step of evaluating the quality of a feature region containing the pet biometric information in the image, and the evaluation condition for the quality is set differently depending on whether the camera is a front camera that captures an image facing the display direction of the display or a rear camera provided on the opposite side of the front camera. Claim 11 An electronic device for conditionally detecting human behavioral information and pet biometric information together, comprising: a camera for capturing an image; a display for visually displaying the image; a processor functionally coupled with the display and the camera; and a memory functionally coupled with the processor, wherein the processor is configured to capture an image of the human and pet through the camera, output the image through the display, extract human behavioral information and pet biometric information from the image, determine the suitability of the image based on the human behavioral information and pet biometric information, and if the image is determined to be suitable, store the behavioral information and biometric information together with the image in memory, wherein the processor determines whether the human behavioral information extracted from the image satisfies a pre-set valid condition, wherein the valid condition includes the human’s gaze being directed forward or making a pre-set facial expression, wherein if the human behavioral information satisfies the valid condition, perform a quality evaluation on a feature region containing the pet biometric information, and if the result of the quality evaluation satisfies a standard, determine the image to be suitable. Claim 12 An electronic device according to claim 11, wherein the camera comprises: a front camera that captures an image facing the display direction of the display; and a rear camera provided on the opposite side of the front camera, and the processor is configured to primarily capture an image of the person and pet with the front camera. Claim 13 An electronic device according to claim 12, wherein the processor is configured to capture the image using a rear camera provided on the opposite side of the front camera in place of the front camera when activation of the front camera is impossible. Claim 14 In paragraph 13, the activation capability of the front camera is determined based on attribute information of the front camera, and the attribute information includes whether manual focus is supported, shooting format, and shooting size, in an electronic device. Claim 15 delete Claim 16 An electronic device according to claim 11, wherein if the facial expression of the person is determined to be a smiling expression or a neutral expression, the image is determined to be suitable, and if the facial expression of the person is determined to be an expression other than a smiling expression or a neutral expression, the image is determined to be unsuitable. Claim 17 An electronic device according to claim 11, wherein if the direction of the person's gaze is determined to be a frontal direction, the image is determined to be suitable, and if the direction of the person's gaze is determined to be other than a frontal direction, the image is determined to be unsuitable. Claim 18 In claim 11, the processor is an electronic device configured to detect a feature region containing the biometric information of the pet and to evaluate the quality of the image of the feature region. Claim 19 In paragraph 18, the above quality is an electronic device including clarity, brightness, and size. Claim 20 An electronic device for conditionally detecting human behavioral information and pet biometric information together, comprising: a camera for capturing an image; a display for visually displaying the image; a processor functionally coupled with the display and the camera; and a memory functionally coupled with the processor, wherein the processor is configured to capture an image of the human and pet through the camera, output the image through the display, extract human behavioral information and pet biometric information from the image, determine the suitability of the image based on the human behavioral information and pet biometric information, and if the image is determined to be suitable, store the behavioral information and biometric information together with the image in memory, wherein the processor is configured to detect a feature region containing the pet biometric information and evaluate the quality of the image of the feature region, and the evaluation condition for the quality is configured differently depending on whether the camera is a front camera that captures an image facing the display direction of the display or a rear camera provided on the opposite side of the front camera.

Images