Companion animal biological information acquisition method and electronic device for conditionally detecting behavior information about person and biological information about companion animal together

The method and electronic device address the challenge of acquiring companion animal biometric information by capturing and evaluating human and animal data, enabling efficient identification and resolving ownership disputes through conditional detection and image evaluation.

WO2024237410A1PCT designated stage expired Publication Date: 2024-11-21PETNOW
6 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2023/019053
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-12
Filing Date
2023-11-24
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Current methods for acquiring biometric information from companion animals are challenging due to their inability to assume specific postures or use sensors, making it difficult to identify them and resolve ownership disputes, especially in situations where verbal communication is impossible.

Method used

A method and electronic device that simultaneously capture and conditionally detect human behavioral information and companion animal biometric information using a camera, processor, and memory, ensuring suitable images are taken for identification by evaluating facial expressions, gaze direction, and biometric quality, and store this information for correlation and authentication.

Benefits of technology

This approach allows for efficient identification of companion animals and resolves ownership disputes by acquiring high-quality images that link human and animal information, facilitating effective biometric recognition and management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2023019053_21112024_PF_FP_ABST
    Figure KR2023019053_21112024_PF_FP_ABST
Patent Text Reader

Abstract

The present invention provides a method for acquiring behavior information about the face and body of a person (owner of a companion animal) together with biological feature information about the companion animal when acquiring the biological feature information about the companion animal using a camera. The companion animal biological information acquisition method of an electronic device for conditionally detecting behavior information about a person together and biological information about a companion animal together according to the present invention, comprises: a step for capturing images including the person and the companion animal via a camera included in the electronic device; a step for outputting the images via a display; a step in which a processor extracts the behavior information about the person and the biological information about the companion animal from the images; a step in which the processor determines the suitability of the images on the basis of the behavior information about the person and the biological information about the companion animal; and a step for storing the behavior information and the biological information on a memory when it is determined that the images are suitable.
Need to check novelty before this filing date? Find Prior Art

Description

A method and electronic device for obtaining biometric information of a pet that conditionally detects both human behavioral information and the biometric information of the pet.

[0001] The present invention relates to a method and an electronic device for simultaneously acquiring behavioral information such as facial expressions and direction of gaze of a person raising a pet and biometric information such as nose print or face shape of the pet, depending on whether a specific condition is met.

[0002]

[0003] In modern society, demand for companion animals, which people can rely on emotionally, is increasing. To provide various additional services, such as lifecycle management and health care, there is a growing need to manage companion animal information in a database. This requires individual identification information, and each pet can be individually defined for each object. For example, similar to human fingerprints, a dog's nose print (the shape of its nose wrinkles) is known to vary from individual to individual, and generally, once a dog reaches adulthood (6 months of age), this characteristic remains unchanged for the remainder of its life. Therefore, it can be said that a dog's nose print can be used as a means of identification.

[0004] However, unlike people who can assume specific positions or use various sensors, pets find it difficult to do so. Because they lack verbal communication, methods such as forcing them into specific positions or contacting sensors using physical force or tools like a frame can be considered. However, this is not desirable from the perspective of protecting the rights of pets and potentially causing resistance from pet owners. Furthermore, because it requires someone to secure the pet and someone to operate the device that captures the pet's biometric information, it presents a practical problem: it's difficult to collect biometric information alone.

[0005] As people and animals coexist in a society, there are increasingly instances where information needs to be linked between pets and their owners. For example, when entering a pet care facility or finding a pet at a pet boarding facility (such as a dog hotel), the facility has no way to verify the owner's identity. Furthermore, in the case of abandonment, there are frequent instances of pets being denied ownership.

[0006] The present invention proposes a method for acquiring biometric information of a companion animal using a camera, and for acquiring facial and body behavioral information of a person (companion animal guardian) together with the biometric information of the companion animal.

[0007] By doing so, the present invention ensures that images more suitable for identification or learning are captured when capturing images for biometric identification of companion animals. Furthermore, the present invention facilitates the identification of companion animals and provides a solution to disputes over ownership of companion animals and denials of ownership that may arise in certain circumstances.

[0008] The solutions of the present invention are not limited to those mentioned above, and other solutions not mentioned will be clearly understood by those skilled in the art from the description below.

[0009]

[0010] A method for acquiring biometric information of a pet by an electronic device that conditionally detects behavioral information of a person and biometric information of a pet according to the present invention comprises the steps of: taking an image including the person and the pet through a camera provided in the electronic device; outputting the image through a display; extracting, by a processor, the behavioral information of the person and the biometric information of the pet from the image; determining, by the processor, whether the image is suitable based on the behavioral information of the person and the biometric information of the pet; and storing, in a memory, the behavioral information and the biometric information if the image is determined to be suitable.

[0011] According to an embodiment of the present invention, the step of capturing the image may include a step of capturing the image of the person and the pet with a front camera that captures the image facing the display direction of the display.

[0012] According to an embodiment of the present invention, when the front camera cannot be activated, a step of capturing the image using a rear camera provided on the opposite side of the front camera instead of the front camera may be included.

[0013] According to an embodiment of the present invention, whether the front camera can be activated is determined based on attribute information of the front camera, and the attribute information may include whether manual focus is supported, a shooting format, and a shooting size.

[0014] According to an embodiment of the present invention, the step of determining whether the image is suitable may include a step of determining whether the image is suitable based on a facial expression of the person or a direction of the person's gaze.

[0015] According to an embodiment of the present invention, if the facial expression of the person is determined to be a smiling expression or a neutral expression, the image may be determined to be suitable, and if the facial expression of the person is determined to be an expression other than a smiling expression or a neutral expression, the image may be determined to be unsuitable.

[0016] According to an embodiment of the present invention, the step of determining whether the image is suitable may include a step of evaluating the quality of a feature region in the image that includes the biometric information of the companion animal.

[0017] According to an embodiment of the present invention, the quality may include clarity, brightness, and size.

[0018] According to an embodiment of the present invention, the evaluation conditions for the quality may be set differently depending on whether the camera is a front camera that takes images facing the display direction of the display or a rear camera provided on the opposite side of the front camera.

[0019] An electronic device for conditionally detecting behavioral information of a person and biometric information of a companion animal according to the present invention comprises: a camera for capturing an image; a display for visually displaying the image; a processor functionally connected to the display and the camera; and a memory functionally connected to the processor. The processor is configured to capture an image of the person and the companion animal through the camera, output the image through the display, extract the behavioral information of the person and the biometric information of the companion animal from the image, determine whether the image is suitable based on the behavioral information of the person and the biometric information of the companion animal, and store the behavioral information and the biometric information together with the image in the memory when the image is determined to be suitable.

[0020] According to the present invention, in the process of acquiring an image for biometric recognition of a companion animal, an image more suitable for learning or identification can be acquired by performing a quality evaluation of the image reflecting human behavioral information to determine whether or not to acquire the image.

[0021] Additionally, human behavioral information and pet biometric information can be acquired simultaneously, ensuring temporal and spatial simultaneity. This allows for a correlation between human behavioral information and pet biometric information, enabling a complex authentication process.

[0022] Figure 1 illustrates the appearance of an electronic device according to the present invention.

[0023] Figure 2 is a block diagram showing the configuration of an electronic device according to the present invention.

[0024] Figure 3 illustrates a system including an electronic device that acquires biometric information of a companion animal and a server that processes the biometric information.

[0025] Figure 4 is a flowchart showing a method for acquiring pet biometric information by conditionally detecting human behavioral information and pet biometric information together.

[0026] Figure 5 is a diagram showing the process of acquiring human behavioral information and pet biometric information.

[0027] Figures 6 and 7 illustrate examples of screens displayed during the process of acquiring human behavioral information and pet biometric information.

[0028] Figures 8 and 9 are flowcharts showing the process of selecting a camera to capture images for acquiring human behavioral information and pet biometric information.

[0029] Figure 10 illustrates an example of an interface for selecting a camera to perform video capture.

[0030] Figure 11 is a flowchart showing the process of checking the compatibility between the behavioral information of a person (HM) and the biometric information of a companion animal.

[0031] Figure 12 is a flowchart showing the process of checking the suitability of human behavior information.

[0032] Figures 13 and 14 illustrate examples of a shooting guide interface according to the results of a suitability test in the process of acquiring human behavioral information and pet biometric information.

[0033] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. The present invention may be implemented in various different forms and is not limited to the embodiments described herein.

[0034] In order to clearly explain the present invention, parts that are not related to the description are omitted, and the same reference numerals are used for identical or similar components throughout the specification.

[0035] In addition, in various embodiments, components having the same configuration are described only in representative embodiments using the same symbols, and in other embodiments, only configurations different from the representative embodiments are described.

[0036] Throughout the specification, when a part is said to be "connected (or coupled)" with another part, this includes not only cases where it is "directly connected (or coupled)" but also cases where it is "indirectly connected (or coupled)" with other elements in between. Furthermore, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0037] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0038] Hereinafter, a method for acquiring biometric information of a companion animal (PT) and an electronic device (1) for performing the same, which conditionally detects behavioral information of a person (HM) and biometric information of a companion animal (PT) according to an embodiment of the present invention, will be described. The biometric information of a companion animal (PT) is acquired to confirm the identity of the companion animal (PT). The biometric information of the companion animal (PT) may include an iris, facial shape, body shape, and nose wrinkle shape (nasal print). This document focuses on photographing the nose print of a dog and the face of a cat as biometric information of the companion animal (PT), but the scope of the companion animal (PT) in the present invention is not limited to dogs and cats, and various physical characteristics of the companion animal (PT) may be used as identification information, and are not limited to the nose print.

[0039] According to the present invention, an electronic device (1) photographs a companion animal (PT) using a camera (10) and acquires biometric information of the companion animal (PT) from the photographed image. The electronic device (1) may be a mobile device such as a smartphone or tablet. In addition, the electronic device (1) may correspond to a computing device (e.g., a laptop computer, a desktop computer). In the case of a foldable mobile device such as a foldable phone, a front camera (11) may be present before and after folding. In the electronic device (1) according to the present invention, the front camera (11) may be provided inside or outside the display (20).

[0040] Fig. 1 illustrates an example of an electronic device (1) in which a front camera (11) is mounted inside a display area. Fig. 1 (a) shows the front (F) of the electronic device (1) in which the front camera (11) is mounted inside the display area, and Fig. 1 (b) shows the rear (R) of the electronic device (1). The exterior of the electronic device (1) may be composed of a frame, a display (20), and a camera (10) (front camera (11), rear camera (12)). In Fig. 1, the front camera (11) may be configured inside the screen area of ​​the display (20). The front camera (11) and the rear camera (12) may each be configured with multiple cameras. Meanwhile, the front camera (11) may also be mounted outside the display area.

[0041] Referring to FIG. 2, an electronic device (1) according to the present invention includes a camera (10), a display (20), a processor (30), and a memory (40). Optionally, the electronic device (1) may further include a gyro sensor (61), a touch sensor (62), or an illuminance sensor (63). Optionally, the electronic device (1) may further include a communication module (50), a speaker (71), or a microphone (72).

[0042] The display (20) is a device that outputs visual information. The display (20) may include a panel and a control circuit. The panel may be equipped with a touch sensor (62). In particular, the display (20) may output an image (preview image, captured frame) of a companion animal (PT) being photographed. In addition, the display (20) may output an interface that provides information to the user during the process of acquiring biometric information of the companion animal (PT).

[0043] A camera (10) is a device that captures images (still images or videos). The camera may include one or more image sensors, lenses, a camera controller, an image processing processor, and a flash. The camera (10) may include a front camera (11) and a rear camera (12). The camera (10) can capture images for acquiring biometric information of a companion animal (PT) and behavioral information of a person (HM). The characteristics of the camera (e.g., focus, exposure) can be controlled so that a characteristic object for acquiring biometric information of a companion animal (PT) can be properly identified. The front camera (11) is a camera that captures images toward the display direction of the display (20), as shown in FIG. 1, and the rear camera (12) is a camera that captures images toward the opposite direction of the display direction of the display (20), as shown in FIG. 2. The camera may include not only a camera built into the electronic device (1), but also an external camera connected to the electronic device (1) by wire or wirelessly.

[0044] The processor (30) can perform calculations and data processing for performing the operation of the electronic device (1), and control each module of the electronic device (1). The processor (30) can execute one or more software programs stored in the memory (40). The processor (30) can include a general-purpose processor (e.g., CPU, AP) that executes an operating system that manages the overall operation of the electronic device (1). In addition, the processor (30) can include a dedicated processor (e.g., GPU, NPU) for specific calculation processing. The processor (30) can include one or more processing circuits. The processor (30) can perform data processing and control operations for acquiring biometric information of a companion animal (PT) and behavioral information of a person (HM).

[0045] The memory (40) can store programs such as an operating system and applications for the operation of the electronic device (1), and data used during the execution of the programs. The memory (40) can include volatile memory (e.g., DRAM) and non-volatile memory (e.g., flash memory). The memory (40) can store images for biometric identification of a companion animal (PT). In addition, the memory (40) can store biometric information of the companion animal (PT) and behavioral information of the person (HM) together with the images.

[0046] The communication module (50) provides an environment in which the electronic device (1) can communicate with an external entity (e.g., a server, a base station, an IoT device, a mobile device). The communication module (50) may include an antenna, an RF circuit, and a communication processor. The communication module (50) may provide data received from an external entity to the processor (30). The communication module (50) may convert the data provided from the processor (30) and transmit it to the external entity. The communication module (50) may receive image data for biometric authentication of a companion animal (PT) provided from the processor (30) and transmit the image data to the server (2). As illustrated in FIG. 3, the electronic device (1) may communicate with the server (2). The electronic device (1) may acquire biometric information of the companion animal (PT) and transmit the acquired biometric information to the server (2). The server (2) may store the biometric information in a database. The server (2) may perform data processing for learning or identifying the companion animal (PT) from the biometric information. The server (2) can generate identification information of a companion animal (PT) from biometric information and behavioral information of a person (HM), and transmit the identification information of the companion animal (PT) to an electronic device (1).

[0047] An electronic device (1) for acquiring biometric information of a companion animal (PT) according to the present invention includes a camera (10) for capturing an image, a display (20) for visually displaying an image, a processor (30) functionally coupled to the display (20) and the camera (10), and a memory (40) functionally coupled to the processor (30).

[0048] The processor (30) is set to capture an image of a person (HM) and a companion animal (PT) through a camera (10), output the image through a display (20), extract behavioral information of the person (HM) and biometric information of the companion animal (PT) from the image, determine whether the image is suitable based on the behavioral information of the person (HM) and the biometric information of the companion animal (PT), and store the behavioral information and biometric information together with the image in a memory (40) if the image is determined to be suitable.

[0049] FIG. 4 is a flowchart showing a method for acquiring biometric information of a companion animal (PT) that conditionally detects behavioral information of a person (HM) and biometric information of a companion animal (PT). The method for acquiring biometric information of a companion animal (PT) that conditionally detects behavioral information of a person (HM) and biometric information of a companion animal (PT) according to the present invention includes a step (S410) of capturing an image including a person (HM) and a companion animal (PT) through a camera (10) provided in an electronic device (1), a step (S420) of outputting the image through a display (20), a step (S430) of extracting behavioral information of the person (HM) and biometric information of the companion animal (PT) from the image, a step (S440) of determining whether the image is suitable based on the behavioral information of the person (HM) and the biometric information of the companion animal (PT), and a step (S450) of storing the behavioral information and biometric information in a memory (40) if the image is determined to be suitable. The video capturing, human behavioral information and animal bio-information extraction, and suitability testing according to the present invention may be performed for each frame, or may be repeatedly performed at a certain frame interval or a certain time interval.

[0050] According to the present invention, when capturing an image for biometric identification of a companion animal (PT), the behavioral information of a human (HM) is also considered in the process of determining whether the image is suitable for recognition or learning. For example, if the person (HM) has a bewildered expression or does not look straight ahead, there is a high possibility that the person is not prepared for video capture with the companion animal (PT). If the person is not prepared for video capture, the companion animal (PT) is also likely to not look straight ahead or move its head, and in this state, even if a video is captured, the possibility of capturing an image of suitable quality is very low.

[0051] Therefore, in the process of determining whether to acquire an image, the behavioral information of the person (HM) is analyzed, and it is determined whether the behavioral information satisfies the conditions for image acquisition, and if the behavioral information of the person (HM) satisfies the conditions for image acquisition and other acquisition conditions, the image is determined to be suitable for biometric identification of the companion animal, and the image can be stored.

[0052] Additionally, a video of a pet (PT) and a person (HM) together can be used to identify the owner of the pet (PT). Therefore, when capturing a video for biometric identification of a pet (PT), by capturing both the animal (PT) and the person (HM), information about the owner can be used to identify the pet (PT).

[0053] According to the present invention, in the process of acquiring an image for biometric identification of a companion animal (PT), in determining whether the image is suitable for identification of the companion animal (PT), the suitability of the feature objects (e.g., face, nose, eyes, etc.) for biometric identification of the companion animal (PT) and the behavioral information of the photographed person (HM) are taken into consideration together. When the companion animal (PT) and the person (HM) are photographed together, a suitability test for the feature objects of the companion animal (PT) and the feature objects of the person (HM) can be performed simultaneously to determine whether the photographed image is suitable for identification of the companion animal (PT). The test for the feature objects of the companion animal (PT) can be performed by examining the clarity, brightness, and size of the image including the feature objects. The test for the feature objects of the person (HM) can be similar to the test for the feature objects of the companion animal (PT). In addition, the test for the feature objects of the person (HM) includes a process of confirming that the person (HM) is taking an image for biometric identification of the companion animal (PT) through the expression and gaze of the person (HM).

[0054] In step S410, an application for biometric identification of a companion animal (PT) is executed, and a camera (10) is activated to initiate video recording. As the camera (10) for video recording, a front camera (11) or a rear camera (12) may be used. As shown in FIG. 5, a front camera (11) suitable for capturing a picture of a person (HM) and a companion animal (PT) together may be preferentially used. That is, the step of capturing a picture (S410) may include a step of capturing a picture of a person (HM) and a companion animal (PT) with a front camera (11) that captures a picture in the display direction of the display (20).

[0055] The process of acquiring biometric information of a companion animal using the front camera (11) can be performed by having the user (HM) take a picture of the companion animal (PT) while holding the electronic device (1), as illustrated in FIG. 5. Compared to using the rear camera (12), there are several advantages to using the front camera (11) to capture biometric information of a companion animal (PT). In general, companion animals such as dogs do not recognize that they are being photographed, so they do not look at the camera but look in another direction. In addition, most companion animals cannot remain still and have a habit of constantly moving their heads, making it very difficult to take a frontal picture suitable for biometric identification. In addition, since most users are not skilled at taking pictures, it is difficult for most users to take an image suitable for biometric identification of a companion animal.

[0056] On the other hand, when taking a picture of a pet using the front camera (11) as shown in Fig. 5, the user can check the image of the pet currently being taken in real time, so an image more suitable for biometric identification of the pet can be acquired. In addition, the user can take the picture while holding the pet, and since the picture is taken in a more comfortable state for both the user and the pet, an image more suitable for biometric identification can be acquired.

[0057] FIGS. 6 and 7 illustrate examples of screens displayed during the process of acquiring behavioral information of a person (HM) and biometric information of a companion animal (PT). FIG. 6 illustrates a screen in which a person and a dog are photographed together, and FIG. 7 illustrates a screen in which a person and a cat are photographed together. When a person and a dog are photographed together as in FIG. 6, the dog's nose is used as a feature object, and an animal feature region (100) including the feature object is extracted. In addition, a person's face is used as a feature object, and a person feature region (200) including the feature object is extracted. When a person and a cat are photographed together as in FIG. 7, the cat's face is used as a feature object, and an animal feature region (100) including the feature object is extracted. In addition, a person's face is used as a feature object, and a person feature region (200) including the feature object is extracted.

[0058] In order to obtain biometric information of a companion animal, the front camera (11) can capture the face of the companion animal. Prior to capturing the image of the front camera (11), the processor (30) can determine whether to activate the front camera (11). That is, the processor (30) can determine whether to capture an image for obtaining biometric information of the companion animal using the front camera (11). If the front camera (11) can be activated, the processor (30) can activate the front camera (11). If the front camera (11) cannot be activated, the processor (30) can activate the rear camera (12) provided on the opposite side of the front camera (11) instead of the front camera (11).

[0059] As illustrated in FIG. 8, the step (S410) of shooting an image including a person (HM) and a pet (PT) includes the steps of checking the attribute information of the front camera (11) (S805), determining whether the front camera (11) can be activated based on the attribute information (S810), activating the front camera (11) if the front camera (11) can be activated (S815), and activating the rear camera (12) provided on the opposite side of the front camera (11) instead of the front camera (11) if the front camera (11) cannot be activated (S820).

[0060] In another embodiment, the user may be allowed to select which camera to use. According to the present invention, the processor (30) checks the attribute information of the front camera (11) and determines whether the front camera (11) can be activated based on the attribute information of the front camera. The attribute information of the front camera (11) may include whether manual focus is supported, a shooting format, a shooting size, and hardware level information. The attribute information of the front camera (11) may be stored as data of a camera management program. For example, the processor (30) may determine that the front camera (11) is a camera that can be used to acquire biometric information of a companion animal if the front camera (11) is a camera that supports manual focus, the shooting format and shooting size satisfy preset conditions, and the hardware level of the camera satisfies a reference level.

[0061] The processor (30) can output information about the properties of the front camera (11) and whether the front camera (11) can be activated through the display (20). By displaying information about the properties of the front camera (11) and whether the front camera (11) can be activated, the user can check whether the front camera (11) can be used to acquire biometric information of a companion animal.

[0062] If the front camera is not activable, the processor (30) can activate the rear camera (12). That is, the processor (30) can perform photography for biometric identification of a pet using the rear camera (12) without activating the front camera (11).

[0063] The processor (30) may output an interface that allows the user to select one of the front camera (11) and the rear camera (12) when the front camera (11) is activated. For example, when the front camera is activated, the processor (30) may output an interface that allows the user to select a camera. For example, basically, the rear camera (12) may be activated to display a preview image, and an icon (400) that allows the user to switch cameras may be output at the lower right of the application screen as shown in FIG. 10. When a touch input to the icon (400) is detected by the touch sensor (62), the processor (30) may activate the front camera (11) and display an image captured by the front camera (11) as a preview image on the display (20). When a touch input to the icon (400) is detected again during the shooting of the front camera (11), the processor (30) may again output an image captured by the rear camera (12) as a preview image on the display (20). As another example, a list could be output that allows the user to select either the front camera (11) or the rear camera (12) before starting shooting.

[0064] When the front camera (11) is selected, the processor (30) activates the front camera (11) to acquire an image for biometric identification of the pet. When the rear camera (12) is selected, the processor (30) activates the rear camera (12) to acquire an image for biometric identification of the pet.

[0065] As illustrated in FIG. 9, in the step (S410) of shooting an image including a person (HM) and a pet (PT), the step (S905) of checking the attribute information of the front camera (11) and the step (S910) of determining whether the front camera (11) can be activated based on the attribute information of the front camera (11) are included. In addition, the step (S940) of outputting the attribute information of the front camera (11) and whether it can be activated through the display (20) may be executed. If the front camera (11) can be activated, the step (S915) of outputting an interface for selecting one of the front camera (11) and the rear camera (12) may be executed. A step (S920) of determining whether a front camera (11) is selected through an interface, a step (S925) of activating the front camera (11) if the front camera (11) is selected, and a step (S930) of activating the rear camera (12) if the front camera (11) is not selected are executed. Thereafter, a process of taking an image for biometric identification of a companion animal through the selected camera can be executed.

[0066] When a video is captured, the biometric information of the companion animal (PT) and the behavioral information of the human (HM) are extracted from the frames of the captured video. The biometric information of the companion animal (PT) can be extracted from the feature region (100) of the companion animal (PT), and the behavioral information of the human (HM) can be extracted from the feature region (200) of the human (HM). As a condition for extracting the biometric information of the companion animal (PT), the quality of the feature region (100) and the suitability for the behavioral information of the human (HM) can be determined.

[0067] According to the present invention, when determining whether a captured image is suitable for animal biometric identification, a quality check of the animal feature region (100) and an inspection of the human feature region (200) are performed simultaneously. The inspection of the human feature region (200) may include quality checks such as clarity, size, and brightness, as well as an inspection of the human (HM) behavioral information. The human (HM) behavioral information includes facial expression information and gaze information.

[0068] Fig. 11 is a flowchart illustrating a process for checking the compatibility of behavioral information of a human (HM) and biometric information of a companion animal (PT). As illustrated in Fig. 11, when an image is captured, a step (S1110) for checking the compatibility of behavioral information of a human (HM) and a step (S1120) for checking the compatibility of biometric information of a companion animal (PT) may be executed.

[0069] The process of performing a suitability check on the behavioral information of a person (HM) may include a step of determining whether the image is suitable based on the facial expression or gaze direction of the person in the image.

[0070] Figure 12 is a flowchart illustrating a process for checking the suitability of behavioral information of a person (HM). Referring to Figure 12, the step (S1110) of performing a suitability check on the behavioral information of a person (HM) includes a step (S1205) of analyzing the facial expression of the person (HM) in the image, and a step (S1210) of determining whether the facial expression is an appropriate expression (smiling or expressionless).

[0071] In the step (S1205) of analyzing the facial expression of a person (HM), a process of extracting a feature region (200) including a region corresponding to the person (HM) and an object capable of identifying the facial expression of the person (HM) from the image is performed. Facial expression information (neutral expression, smile, embarrassment, anger, sadness) for the facial region of the person (HM) can be extracted as the person's action information. Various methods can be used to analyze the facial expression of the person (HM), including algorithms such as Eigenfaces, LBP (Local Binary Patterns), CNN (Convolutional Neural Networks), and SVM (Support Vector Machines).

[0072] If the facial expression is not an appropriate expression (e.g., an angry or embarrassed expression), the video containing the face is determined to be unsuitable (S1230). For example, as illustrated in FIG. 13, if a person's face is obscured and the expression is not detected, or if an embarrassed expression is detected, the video is determined to be unsuitable, and a guidance message (320) instructing the person (HM) to smile may be output.

[0073] If the facial expression is an appropriate expression, a step of analyzing the gaze direction of the person (HM) (S1215) and a step of determining whether the gaze direction of the person is frontal (S1220) are performed. In the step of analyzing the gaze direction of the person (HM) (S1215), an eye within the face region of the person may be detected as a feature object, and an algorithm for detecting the gaze direction from an eye image may be executed. For example, the gaze direction may be detected by detecting the position of the pupil within the eye region and comparing the position of the eye with the position of the pupil. An algorithm such as PCCR (Pupil Center Corneal Reflection) may be used to detect the gaze direction.

[0074] If the gaze direction is determined to be frontal, the captured image is determined to be suitable (S1225). If the gaze direction is determined to be other than frontal, the captured image is determined to be unsuitable (S1230). For example, as illustrated in FIG. 13, if the gaze direction of a person is not frontal but upward, a guidance message (320) may be output to have the person (HM) look at the front of the camera. On the other hand, as illustrated in FIG. 14, if the person's expression is a smiling expression and the gaze direction is frontal, the image may be determined to be suitable. In this case, a guidance message (300) may be output to continuously maintain the same expression and gaze.

[0075] In Figure 12, expression analysis is performed first and then gaze analysis is performed as an example. However, gaze analysis may be performed first, or expression analysis and gaze analysis may be performed in parallel. Alternatively, only one of expression analysis and gaze analysis may be used.

[0076] Meanwhile, after an image of a feature object for biometric recognition of a companion animal is acquired through the front camera (11) or the rear camera (12), a quality evaluation may be performed to determine whether the image of the feature object is suitable for learning or identifying the companion animal. According to the present invention, the processor (30) may evaluate the quality of the image of the feature object. Conditions defined to evaluate the quality of the image of the feature object may include clarity, brightness, and size. The quality evaluation conditions may differ depending on whether the camera that captured the image of the feature object is the front camera (11) or the rear camera (12). For example, when judging the image quality through the size of the image of the feature object, the size condition of the image captured by the front camera (11) may be set to be larger than that of the rear camera (12). In general, the front camera (11) is configured as a camera with lower performance than the rear camera (12). For example, the front camera (11) may have lower overall image quality than the rear camera (12) and may not support the OIS (Optical Image Stabilization) function, which is a hand shake prevention function. Therefore, when an image is captured through the front camera (11), the quality evaluation of the image of the characteristic object can be performed under a higher level of quality conditions. Therefore, even when captured through the front camera (11), the biometric identification of the pet can be smoothly performed through the image of the characteristic object.

[0077] The results of a suitability test for the biometric information of a companion animal (PT) in a captured video can be output through a screen. If the companion animal (PT) in the captured video is identified as a dog and the quality evaluation result for the feature area (200) including the feature object of the companion animal (PT) is determined to be unsuitable, a message (310) guiding the dog to look at the front of the camera can be output as illustrated in FIG. 13. If the quality evaluation result for the feature area (200) including the feature object of the companion animal (PT) is determined to be suitable, a message (300) guiding the dog to maintain the corresponding posture can be output as illustrated in FIG. 14.

[0078] If both the behavioral information of the person (HM) and the biometric information of the companion animal (PT) are determined to be suitable in the video, the captured video is stored in the memory (40). The video stored in the memory (40) can be transmitted to the server (2) via the communication module (50).

[0079] When the processor (30) of the electronic device (1) transmits an image of a feature object to the server (2), it can transmit, together with the image data, related information such as behavioral information of a person (HM), identity information, and information about the camera that captured the image. The server (2) can store / manage the behavioral information and identity information of the person (HM) in association with the identity information of the companion animal (PT). In addition, the server (2) can know which camera was used to capture the image through the camera information. The server (2) can determine a method of processing the feature object through the camera information. That is, the server (2) can process the image differently depending on whether the camera that captured the image is the front camera (11) or the rear camera (12).

[0080] The present embodiments and drawings attached to the present specification only clearly illustrate a part of the technical idea included in the present invention, and it will be obvious that all modified examples and specific embodiments that can be easily inferred by a person skilled in the art within the scope of the technical idea included in the specification and drawings of the present invention are included in the scope of the rights of the present invention.

[0081] Therefore, the idea of ​​the present invention should not be limited to the described embodiments, and all things that are equivalent or equivalent to the claims described below as well as the claims are considered to fall within the scope of the idea of ​​the present invention.

Claims

1. In a method for acquiring biometric information of a pet by an electronic device that conditionally detects human behavioral information and biometric information of a pet, A step of taking a video including the person and the pet through a camera provided in the electronic device; A step of outputting the above image through a display; A step of extracting human behavioral information and pet biometric information from the above video; A step of determining whether the image is suitable based on the behavioral information of the person and the biometric information of the pet; and A method for acquiring biometric information of a companion animal, comprising the step of storing the behavioral information and the biometric information in memory when the above image is determined to be suitable.

2. In paragraph 1, A method for acquiring biometric information of a pet, wherein the step of capturing the image first includes a step of capturing an image of the person and the pet with a front camera that captures the image facing the display direction of the display.

3. In paragraph 2, A method for acquiring biometric information of a companion animal, comprising the step of taking the image using a rear camera provided on the opposite side of the front camera instead of the front camera when the front camera cannot be activated.

4. In paragraph 3, Whether the front camera can be activated is determined based on the attribute information of the front camera, The above property information includes a method for acquiring pet biometric information, including whether manual focus is supported, shooting format, and shooting size.

5. In paragraph 1, The step of determining whether the above video is suitable is: A method for acquiring biometric information of a companion animal, comprising a step of determining whether the image is suitable based on the facial expression of the person or the direction of the person's gaze.

6. In paragraph 5, If the facial expression of the person above is judged to be a smiling expression or a neutral expression, the image above is judged to be suitable. A method for acquiring biometric information of a companion animal, wherein the video is judged to be inappropriate if the facial expression of the person is judged to be other than a smiling expression or a blank expression.

7. In paragraph 5, If the direction of the person's gaze is determined to be frontal, the image is determined to be suitable. A method for acquiring biometric information of a companion animal, wherein the image is determined to be inappropriate if the direction of the gaze of the person is determined to be other than the frontal direction.

8. In paragraph 1, A method for obtaining biometric information of a companion animal, wherein the step of determining whether the image is suitable includes a step of evaluating the quality of a feature region containing the biometric information of the companion animal in the image.

9. In paragraph 8, The above quality is a method for acquiring pet biometric information including clarity, brightness, and size.

10. In paragraph 8, A method for acquiring pet biometric information, wherein the evaluation conditions for the above quality are set differently depending on whether the camera is a front camera that takes images facing the display direction of the display or a rear camera provided on the opposite side of the front camera.

11. In an electronic device that conditionally detects human behavioral information and pet biometric information, A camera that records video; A display visually displaying the above image; a processor functionally coupled with the display and the camera; and comprising a memory functionally coupled to the processor; The above processor, Take a video of the person and pet through the above camera, Output the above image through the display, Extract human behavioral information and pet biometric information from the above video, Based on the above-mentioned person's behavioral information and the pet's biometric information, the suitability of the above-mentioned video is determined, An electronic device configured to store the behavioral information and the biometric information together with the image in memory when the image is determined to be suitable.

12. In paragraph 11, The above camera, A front camera that takes images facing the display direction of the above display; and Including a rear camera provided on the opposite side of the front camera, An electronic device wherein the processor is set to primarily capture images of the person and the pet using the front camera.

13. In paragraph 12, An electronic device wherein the processor is set to capture the image using a rear camera provided on the opposite side of the front camera instead of the front camera when the front camera cannot be activated.

14. In paragraph 13, Whether the front camera can be activated is determined based on the attribute information of the front camera, The above property information includes whether manual focus is supported, shooting format, and shooting size of the electronic device.

15. In paragraph 11, An electronic device wherein the processor is set to determine whether the image is suitable based on the facial expression of the person or the direction of the person's gaze.

Citation Information

Patent Citations

  • Detection device, display control device and imaging control device provided with the detection device, object detection method, control program, and recording medium

    JP2013065110A

  • Mobile terminal and method for controlling the same

    KR1020150133466A

  • Solar-powered smart wallet

    KR1020210046973A

  • Electrode Notching Apparatus and Electrode Notching Method Using the same

    KR1020240035106A

  • Food waste collection device to prevent infectious diseases

    KR102462389B1