Door image analysis based artificial intelligence secretary device and system

Abstract

One aspect of the present invention discloses a door video analysis-based AI (Artificial Intelligence) secretary device. The device includes a video receiver that receives video captured by a camera attached to a door (the video includes video of people entering and exiting the door), a repository that stores the received video, and a processor that uses an artificial intelligence (AI) model to analyze the video stored in the repository to infer characteristics of the people entering and exiting the door and provide information corresponding to the inferred characteristics.

Description

[Technical field]

[0001] The present invention relates to an artificial intelligence (AI) secretary device, and more particularly to an AI secretary device for realizing a smart home. [Background technology]

[0002] Recently, most households have Internet of Things-based home automation devices in their homes. Most home automation devices perform operations for the convenience of users and therefore may have the function of checking the status of users.

[0003] However, in order to check the user's condition, the user must take separate actions such as taking a separate questionnaire survey or taking a separate video, which is inconvenient. Also, the amount of information acquired in this manner is not large, and the types of information are not diverse, so there are problems in that accurate checks and the information provided therewith are not appropriate. Summary of the Invention [Problem to be solved by the invention]

[0004] In order to solve the above-mentioned problems, an object of one aspect of the present invention is to provide an AI secretary device that analyzes images captured by a door camera and provides the user with necessary information based on an artificial intelligence model. [Means for solving the problem]

[0005] To achieve the above-mentioned object, according to one aspect of the present invention, a door image analysis-based AI (Artificial Intelligence) secretary device includes an image receiving unit that receives images captured by a camera attached to a door (the images include images of people entering and exiting the door), a repository that stores the received images, and a processor that uses an artificial intelligence (AI) model to analyze the images stored in the repository to infer characteristics of the people entering and exiting the door and provide information corresponding to the inferred characteristics.

[0006] The processor may infer at least one of a body condition, a body shape, and a body size.

[0007] The processor may recommend products that correspond to the inferred characteristics.

[0008] The processor may infer a physical health state and utilize a physical diet model to recommend a diet menu or products corresponding to the inferred physical health state.

[0009] The processor may infer a body type or size and recommend corresponding clothing or merchandise based on the inferred body type or size.

[0010] The device may further include a display device for visually displaying the corresponding information and a speaker for audibly outputting the corresponding information.

[0011] The display device is a touch screen, and the noted information can be displayed to the person entering or leaving through the touch screen.

[0012] The processor can analyze objects in the image to identify people in the image, the processor can identify people associated with the memo information, and when a person corresponding to the identified memo information enters or exits the door, display memo information corresponding to the person.

[0013] The device may further include a motion sensor for detecting a person moving, and the processor may derive the corresponding information when a movement is detected via the motion sensor.

[0014] The processor can derive corresponding information when the motion is sensed by identifying the person causing the motion and inferring characteristics of the identified person.

[0015] The AI ​​secretary device communicates with a server device associated with an application executed on a user terminal, and the AI ​​secretary device can provide the corresponding information to the user terminal when the application is executed on the user terminal.

[0016] The server device can communicate via a cloud-based network, a home network, or a D2D (Device to Device) method, and information in the repository can be transferred to the server device.

[0017] The processor can identify a first and a second person by respectively identifying the first and second person, providing corresponding first information to the first person and providing corresponding second information to the second person, and determining whether the first and second people enter and exit the door by a threshold value or more.

[0018] The processor analyzes the stored video to infer physiognomy information and fortune-telling information of the person entering or leaving the building, and the physiognomy information and fortune-telling information can be inferred using an oriental science solution.

[0019] The processor may provide advertising content to the display device for display on the display device, and the display device may display the advertising content received from the processor.

[0020] At least one of the display device and the speaker may be provided in a kiosk configuration.

[0021] The device further includes a microphone for capturing audio information of the entering and exiting persons, and the processor can use the audio information captured via the microphone to infer characteristics of the entering and exiting persons.

[0022] The artificial intelligence model can link with a server that manages criminal identity information, medical information, and insurance information to provide information related to the people entering and leaving the facility using the criminal identity information, medical information, and insurance information.

[0023] The banking system is connected to an online banking server associated with the user's account and can provide online banking functions using the account.

[0024] The processor can be in conjunction with a wearable device attached to the person entering or exiting the facility to measure the person's health condition, and can provide medical and insurance-related information relevant to the person using information obtained from the wearable device.

[0025] The wearable device can obtain at least one of the following information: average blood oxygen saturation and average heart rate wave value and provide it to the processor.

[0026] The processor may analyze objects in the stored image from dynamic data, the dynamic data including at least one of animation, light sources, and graphs.

[0027] The data analyzed by the processor may include at least one of the body shape, size, shadows over time, light reflection of the body by a light source, skeleton, head shape, hair, hand shape, fingerprints, palm lines, pores, blood vessels, retina, iris, face, body temperature, walking style, voice, and sound waves of the person entering or exiting.

[0028] The processor can utilize at least one of information about the person at an SNS server and an application server and information stored in memory of a device carried by the person to more accurately infer characteristics of the person.

[0029] The processor can analyze palm reading information of the visitors to infer fortune information of the visitors.

[0030] According to another aspect of the present invention for achieving the above object, a door image analysis-based AI (AI: Artificial Intelligence) secretary system includes a camera attached to a door for capturing images of people entering and exiting, an AI secretary device for inferring characteristics of the people entering and exiting by analyzing images captured by the camera using an artificial intelligence model and providing information corresponding to the inferred characteristics to an output device, and an output device for outputting the information provided by the AI ​​secretary device. Effect of the Invention

[0031] According to the door image-based AI secretary device of the present invention, since information about the user is accumulated through a camera attached to the door that the user enters and exits every day, without the need for separate video shooting or input of user status information, it is possible to provide the user with more accurate information as required, and since it does not require the user to make an effort to provide information, it has the effect of providing ease of use. [Brief description of the drawings]

[0032] [Figure 1] 1 is a conceptual diagram showing a system including an AI secretary device based on door image analysis according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a detailed block diagram illustrating the configuration of an AI secretary device based on door image analysis according to an embodiment of the present invention. [Diagram 3] 4 is a flowchart illustrating an operation method of an AI secretary device based on door image analysis according to an embodiment of the present invention. [Figure 4]1 is a conceptual diagram illustrating a process in which a door image analysis-based AI secretary device outputs guidance information by detecting motion according to an embodiment of the present invention. FIG. [Diagram 5] FIG. 2 is a conceptual diagram illustrating how a door image analysis-based AI secretary device according to an embodiment of the present invention works with an application or a cloud server. [Figure 6] 1 is a flowchart illustrating a process in which an AI secretary device based on door image analysis identifies people entering and exiting a door based on image analysis according to an embodiment of the present invention. [Figure 7] 1 is a conceptual diagram illustrating a process in which an AI secretary device based on door image analysis according to an embodiment of the present invention displays advertisement content in conjunction with a door lock device. [Figure 8] FIG. 2 is a conceptual diagram illustrating a process in which a door image analysis-based AI secretary device according to an embodiment of the present invention interfaces with an external server. [Figure 9] FIG. 2 is a block diagram illustrating an AI secretary device based on door image analysis according to an embodiment of the present invention working with a wearable device attached to a user. [Figure 10] FIG. 2 is a conceptual diagram illustrating a memo function provided by an AI secretary device based on door image analysis according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] The present invention can be modified in various ways and can have various embodiments, so that a specific embodiment will be illustrated in the drawings and described in detail.

[0034] However, it should be understood that this is not intended to limit the invention to the particular embodiment, but rather to include all modifications, equivalents, or alternatives falling within the spirit and scope of the invention.

[0035] Terms such as "first", "second", etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only to distinguish one component from another. For example, a first component may be named a second component, and similarly, the second component may be named a first component, without departing from the scope of the invention. The term "and / or" includes a combination of multiple associated listed items or any of multiple associated listed items.

[0036] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that the component may be directly coupled or connected to the other component, but there may be other components in between. In contrast, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no other components in between.

[0037] The terms used in this application are merely used to describe certain embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this application, the terms "include" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0038] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms as defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as having an ideal or overly formal meaning unless expressly defined in this application.

[0039] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. In the description of the present invention, the same reference numerals are used for the same components in the drawings to facilitate overall understanding, and duplicate descriptions of the same components will be omitted.

[0040] 1 is a conceptual diagram showing a system including an AI secretary device based on door image analysis according to an embodiment of the present invention. As shown in FIG. 1, the AI ​​secretary system according to an embodiment of the present invention includes a network 100, door shooting cameras 110-1 and 110-2, a door lock device 120, an AI secretary device 130, and an output device 140.

[0041] Please refer to FIG. 1. The door photographing cameras 110-1 and 110-2 photograph people entering and exiting the door. The door photographing cameras 110-1 and 110-2 are installed outside and / or inside the door and can photograph an area outside the door and an area inside the door. The door photographing cameras 110-1 and 110-2 photograph people entering and exiting the door and provide the photographed images to the AI ​​secretary device 130 via the network 100. In one example, the camera usually only photographs people and does not transmit images, but provides images to the AI ​​secretary device 130 only when an object is present in the image. Alternatively, the camera can be set to start photographing and transmitting images only when entering and exiting the door in conjunction with a motion sensor.

[0042] The door lock device 120 controls the opening and closing of the door in cooperation with the AI ​​secretary device 130. The door lock device 120 can be opened and closed by various methods such as a fingerprint recognition method, a button method, a pattern method, an iris recognition method, etc., and can be opened and closed by a push-pull method or a rotary handle method. The door lock device 120 can also be connected to the AI ​​secretary device 130.

[0043] The AI ​​secretary device 130 is linked to the door photographing cameras 110-1 and 110-2, the door lock device 120, and the output device 140 via the network 100. The AI ​​secretary device 130 obtains and analyzes image information from the door photographing cameras 110-1 and 110-2 to infer the characteristics of people entering and exiting the building, and provides information corresponding to the inferred characteristics via the output device 140. The AI ​​secretary device 130 may be implemented as a computing device and may include a processor for image analysis and characteristic inference. In one example, the AI ​​secretary device 130 may be a computing device installed in a home. In some cases, the AI ​​secretary device 130 may be referred to as a mobile station (MS), user equipment (UE), user terminal (UT), wireless terminal, access terminal (AT), terminal, fixed or mobile subscriber unit (Subscriber Unit), subscriber station (SS), cellular telephone, wireless device, wireless communication device, wireless transmit / receive unit (WTRU), mobile node, mobile, mobile station, personal digital assistant (PDA), smartphone, laptop, netbook, personal computer, wireless sensor, consumer electronics (CE), or other terminology. In some cases, the AI ​​secretary device 130 may be implemented as a device integrated with the door lock device 120.

[0044] The output device 140 includes a display unit 142 and a speaker 144. This may be realized as one device or as separate devices. The output device 140 may be provided in a form attached to a door. That is, when a person goes out of the door, the display unit 142 attached to the door can recommend products or provide related information to encourage purchases, and the speaker 144 can also operate in a form of speaking guidance information. The display unit 142 may be a touch screen. In some cases, the display unit 142 and the speaker 144 may be realized in the form of a kiosk placed inside a house or in front of a house or at the entrance to an apartment.

[0045] Network 100 includes a short-range network for wireless communication within the home, including at least one of Wi-Fi, Bluetooth, and Zigbee communications. In other examples, network 100 may include broadband and / or wired communications, such as 3G, LTE, and 5G communications.

[0046] 2 is a detailed block diagram showing the configuration of an AI secretary device based on door image analysis according to an embodiment of the present invention. As shown in FIG. 2, the AI ​​secretary device according to an embodiment of the present invention includes a communication unit 210, a memory 220, a processor 230, and an input / output unit 240.

[0047] 2, the communication unit 210 is a component that communicates with an external device via a wired and / or wireless network. The communication unit 210 may include an antenna and / or a communication processor.

[0048] The processor 220 is a component that executes image analysis and characteristic inference functions and additional functions associated therewith, and executes programs or applications related to the corresponding functions. The processor 220 can execute programs related to an artificial intelligence model. The artificial intelligence model may include a video analysis model (object inference model), an audio analysis model, a model for inferring properties of an object, and a model for retrieving and outputting information corresponding to the inferred properties, including at least one of an Artificial Neural Network (ANN), a Deep Neural Network (DNN), a Convolution Neural Network (CNN), and a Recurrent Neural Network (RNN).

[0049] The processor 220 is connected to the memory 230 via a signal line, and executes a program stored in the memory 230 .

[0050] The memory 230 (which may be called a "storage") is a storage device that stores instructions related to a program to be executed by the processor 220 and stores data to be processed by the processor 220. The memory 230 stores image information acquired from the door shooting camera.

[0051] The input / output module 240 includes information input means such as a keyboard and a mouse, and information output means such as a monitor, a television, a touch screen, etc. This may be the same as the output device 140 in FIG. 1, or may be realized as a separate device for the AI ​​secretary device.

[0052] FIG. 3 is a flowchart showing an operation method of an AI secretary device based on door image analysis according to an embodiment of the present invention.

[0053] Please refer to FIG. 3. The device receives video information from the door camera. Then, it analyzes the object present in the video (S310). At this time, it is determined whether the object present in the video is a person, and the state of the person's body and the items attached to the body (e.g., clothes, shoes, and other accessories) is analyzed. The device accumulates and uses video information from the door video to identify and recognize the person entering or exiting the door, and continuously updates the state of the recognized person entering or exiting the door. At this time, information on the shooting time may be provided when the video is collected, and information on the shooting time and other environmental information at the corresponding time may also be analyzed when analyzing the video. As a result, it is possible to introduce information and products corresponding to the current environment and the current time by referring to video information of the same environment and time in the past.

[0054] The device determines the physical condition, body shape, and related items of the person entering or leaving the building from the results of the video analysis (S320). The device infers the status of the person entering or leaving the building by accumulating video data taken by the door camera every day. It determines the current status of the person entering or leaving the building or related items by judging changes in body shape, whether there are any wounds on the body, whether the hair has grown beyond a certain length, etc.

[0055] More specifically, the AI ​​secretary device can process not only static photo data but also dynamic data. The device can analyze objects in the stored video from the dynamic data. In this case, the dynamic data includes at least one of video, light source, and graph. In particular, the device may include any one of the following: body shape and size including wearing various clothes and accessories, shadows over time, light reflection of the body by light source, skeleton, head shape, hand shape, fingerprints (which can be obtained via a door lock device), palm lines, pores, blood vessels, retina, iris, face, body temperature, walking style, voice, and sound waves (which can be obtained via a microphone).

[0056] Meanwhile, in order to more accurately identify who enters and leaves the room based on the door image, the device can accumulate, share, and link the memory (image data in the memory) installed in the user terminal of the identified person, the image storage application, and user information registered in the SNS (Social Network Service) after identifying the person, so that more data can be accumulated and used for analysis. At this time, it is preferable that the devices, applications, SNS, etc. related to the identified person are previously set up so that they can be linked. At this time, the consent to collection, collection period, and items to be collected can be selected in detail through user settings, etc.

[0057] The device provides corresponding information based on the inferred status information (S330). The device can accumulate environmental information and provide guidance on appropriate outfits corresponding to today's weather forecast. For example, if the body shape has changed by more than a reference value, new sizes of clothing and shoes that fit the changed body shape can be recommended. Also, if it is confirmed that the hair or nails have grown by more than a reference value, a hair / nail cut can be recommended. At this time, in conjunction with card company information, it can automatically recommend sending a reservation email to a beauty salon that the visitor previously visited. If the visitor enters only "yes" on the recommendation screen of the AI ​​secretary device, the AI ​​secretary device can automatically send reservation information to the terminal of the corresponding beauty salon.

[0058] Additionally, the health condition of the visitor can be inferred through the analysis of body type and body condition, and a diet menu or related products corresponding to the current body condition can be recommended using a body diet analysis model. As described above, the device can recommend corresponding clothing and products based on the inferred body type or body size. At this time, it is preferable that both information such as body type and current season / weather are taken into consideration. In response to wounds (such as abrasions) on the visitor's body, the corresponding area can be displayed via the output device, and corresponding measures (application of ointment and bandage) can be recommended. In addition, information on purchasing daily necessities necessary for the relevant person can be provided based on the visitor's skin condition, hair condition, etc.

[0059] When there are multiple people entering and exiting the door, the device can identify each person independently, and the information provided can match the information corresponding to the identified person. That is, if the person currently photographed by the door camera is "A", it can provide "X" information using the accumulated data related to "A", and if the current person entering and exiting is "B", it can provide "Y" information using the accumulated data related to "B".

[0060] Meanwhile, the device can provide a personalized shopping function in conjunction with a diet app. During the diet process, the device can recommend the best time to buy suitable clothes by linking with an application or a weight scale that has a BMI (Body Mass Index) measurement function. In some cases, photos of the clothes being worn, product and brand information can be shared with other users who use the AI ​​secretary device, and the users who share the photos can receive advertising revenue according to the shared photos.

[0061] The device can infer the physiognomy and fortune-telling information of the person entering or leaving the room from the door and related images. This is done using an Oriental Studies solution pre-stored in the device. The Oriental Studies solution includes analysis programs related to the Book of Changes / Four Pillars and physiognomy. It can also analyze the palm information of the person entering or leaving the room, match it with today's fortune information, and output the corresponding fortune information.

[0062] Meanwhile, the device acquires voice information of people entering and exiting the door using a voice input means such as a microphone (not shown), and stores and uses the corresponding data to recognize the person entering and exiting the door. That is, not only video information but also voice information can be used together.

[0063] FIG. 4 is a conceptual diagram illustrating a process in which an AI secretary device based on door image analysis outputs guidance information in response to motion detection according to an embodiment of the present invention.

[0064] Refer to Fig. 4. The AI ​​secretary device 420 can constantly output the guidance information determined through the process of Fig. 3, but can also be controlled to output only when a person moves through the corresponding door in consideration of reducing power consumption. The device 420 can be controlled so that the guidance information is output in the form of text, video, or audio display through the output device 430 only when human movement is detected within a pre-set range (e.g., a 3m radius from the door) by the motion detection sensor 410 or a door camera (not shown).

[0065] FIG. 5 is a conceptual diagram illustrating how a door image analysis-based AI secretary device according to an embodiment of the present invention works with an application or a cloud server.

[0066] Please refer to Fig. 5. An AI secretary device 510 can be connected to the Internet via a network 520, and can thereby be linked to a cloud server and / or an application server 530. The server 530 can share information with at least one user terminal 540-1 to 540-N.

[0067] That is, the server 530 may be a server device related to an application executed in the user terminals 540-1 to 540-N, and the AI ​​secretary device 510 may provide guidance information to the user terminals 540-1 to 540-N when the application is executed in the user terminals 540-1 to 540-N. In this case, the server 530 may communicate using a cloud network or a home network. In the case of communication using a home network, the server 530 may be a home server. Or, the communication may be performed by a D2D (Device to Device) method. In order to improve security, the AI ​​secretary device 510 and the user terminals 540-1 to 540-N may directly communicate with each other by a D2D method. Also, at least a part of the information in the memory of the AI ​​secretary device 510 may be transmitted to the server 530 in real time or periodically.

[0068] The user terminals 540-1 to 540-N may be terminals carried by the door users or users related to them (e.g., family members). At this time, the guide information can also be set in advance to identify the door users and provide it only to the terminals corresponding to the identified door users. That is, the information of door user "A" can be set to be transmitted only to terminal 540-1, and the information of door user "B" can be set to be transmitted only to terminal 540-2.

[0069] Also, the AI ​​secretary device 510 may not directly provide information to the user terminals 540-1 to 540-N, but the server 530 may receive guidance information from the AI ​​secretary device 510 in real time, and when there is a request from the user terminals 540-1 to 540-N, the server 530 may directly provide the guidance information corresponding to the request. However, in order to improve security, it is preferable not to store information related to the user in the server 530, but to provide the video analysis information and / or guidance information directly to the user terminals 540-1 to 540-N.

[0070] FIG. 6 is a flowchart illustrating a process in which an AI secretary device based on door image analysis identifies people entering and exiting a door through image analysis according to an embodiment of the present invention.

[0071] See Fig. 6. When the device acquires an image from the door camera (S610), it identifies objects present in the image using an object recognition algorithm (S620). At this time, since analyzing information on visitors who visit the house intermittently and providing related information would waste the processing power of the device, the device determines whether the number of times a particular object has been identified is equal to or exceeds a preset reference value (S630), and only if it is equal to or exceeds the reference value, it determines that the person is a regular resident of the house and infers the characteristics of the person (S640). At this time, the concept of time can also be taken into consideration and the reference value can be set to the number of times the person enters and leaves the house within a reference day such as a week or a month.

[0072] FIG. 7 is a conceptual diagram illustrating a process in which an AI secretary device based on door image analysis according to an embodiment of the present invention displays advertisement content in conjunction with a door lock device.

[0073] Referring to FIG. 7, an AI secretary device (not shown) can be linked to a door lock device 710 as well as a display device 720 attached to the outside of the door. In this case, the AI ​​secretary device can provide advertisement content information to the display device 720 using a network. This includes advertisement information of business entities. For example, the advertisement information includes discount information, coupon information, related product information, etc. of XX restaurants located within a certain radius of the house door to which the display device 720 is attached. The advertisement information may be advertisement information of a business entity within a certain radius of the house, so the business entity (advertiser) providing the advertisement content provides the advertisement content to the AI ​​secretary device via a local network, and the AI ​​secretary device outputs the advertisement content. The advertiser can share a certain portion of the revenue obtained by outputting the advertisement content. The display device 720 can be controlled to display advertisement content only when the approach of a person is detected in conjunction with a motion sensor (or camera).

[0074] On the other hand, in some cases, the display device 720 may be integrally formed with the door lock device 710 .

[0075] FIG. 8 is a conceptual diagram illustrating a process in which an AI secretary device based on door image analysis according to an embodiment of the present invention links with an external server.

[0076] Referring to Fig. 8, the AI ​​secretary device 810 can be linked to an external server 830 via a network 820. Here, the external server 830 may include a bank / card company, an insurance company, an online shopping mall server, a medical institution server, and other server devices.

[0077] The device 810 provides an online banking function by interfacing with a server of a bank / card company associated with the customer's account, thereby enabling the customer to successfully carry out online transactions such as account transfers and payment of expenses. In particular, since the device can also interfacing with an online shopping mall server, if the guidance information is recommended product information, the device can also provide a function of searching for the best price to find an appropriate online shopping mall and purchase the recommended product.

[0078] The device can link with the server of a medical institution or insurance company to obtain information from the database of the institution. As a result, as described above, it is possible to infer and simulate incidents and accidents that may cause physical changes to visitors and their extent of damage. The information from the database can also be used to predict physical changes. In particular, by linking with the server of a medical institution, it is possible to obtain treatment information suitable for the current health condition. For example, when an abrasion is detected, the device searches the medical insurance database and connects to the server of the hospital that the visitor frequently visits, and inquires about treatment at the connected hospital. If the visitor chooses to receive treatment, the device can provide the server with current status information (abrasion present and related video provided) and receive appropriate medical treatment (medical treatment information) from the server. This operation can also be provided to the user terminal in conjunction with the user terminal.

[0079] The device can be linked to the servers of public institutions such as police stations and fire stations to access databases of criminal identity information, etc. The device can compare the video analysis information of people entering and leaving the building with the criminal identity information obtained from the database, output guidance information such as a warning to persons suspected of being criminals, and control the device to take automatic alarm response measures in the form of notifying related institutions.

[0080] FIG. 9 is a block diagram illustrating an AI secretary device based on door image analysis according to an embodiment of the present invention working in conjunction with a wearable device attached to a user.

[0081] Please refer to FIG. 9. The AI ​​secretary device 910 can be linked to a wearable device 920 attached to a user. The wearable device scans, analyzes, and stores physical information including location information from the moment the user starts wearing it, and shares it with the AI ​​secretary device 910. The wearable device 920 can measure average blood oxygen saturation, average heart rate, etc., which can be used individually or in combination as data to authenticate people entering and exiting the door. That is, such data can be used to unlock a door lock device (not shown).

[0082] The device can process the information acquired from the wearable device 920 in conjunction with the medical / insurance company information described in the embodiment of Fig. 8. When the wearable device 920 detects a value that is out of the standard range for the average heart rate, blood oxygen saturation, etc. of the visitor, the device can notify the medical institution server of this. At this time, the insurance company information can be used to check the information of existing visitors.

[0083] FIG. 10 is a conceptual diagram for explaining a memo function provided by an AI secretary device based on door image analysis according to an embodiment of the present invention.

[0084] Referring to FIG. 10, the AI ​​secretary device 1010 can be linked to a display device 1030 attached to the top and / or inside of the door. At this time, the display device 1030 can leave a memo in the form of text or other pictures. The display device 1030 can be realized as a touch screen and can also include an input device function, or a separate keyboard can be arranged nearby to input a memo, or a memo message can be input using the input means of a user terminal or the AI ​​secretary device 1010. The memo message may be a message that is seen by people entering or leaving the door. For example, a mother may leave a message such as "Take XX with you" for her son. In this case, the device 1010 may display the memo at all times, but may control the display device 1030 to display the message when a person matching the message content approaches the door. That is, the device 1010 may analyze video (or audio) via a door camera (or microphone) and control the display device 1030 to display the corresponding memo when it detects the approach of a person who is the subject of the memo message to the door. For this, not only video / audio analysis but also message content analysis may be required. Therefore, the AI ​​secretary device 1010 may use a syntax analysis algorithm to analyze the text of the memo message, extract terms (e.g., mother, father, son, daughter, or their names, nicknames, etc.) that refer to the object contained therein, match the corresponding video, and respond to the approach of a person corresponding to the matched video.

[0085] The device 1010 may store multiple memo messages together, and may control the display device 1010 to identify multiple people and display a first message to person A and a second message to person B at different times. Alternatively, when a group enters or leaves the building, the device 1010 may control the display device 1010 to display a combination of the first and second messages to people A and B at the same time.

[0086] In another example, the memo message may be transmitted to a user terminal of a person who is the subject of the memo message, and data may be transmitted so that the person can review the memo via an application on the user terminal or a text message.

[0087] Although the above description has been given with reference to the drawings and embodiments, it is not intended that the scope of protection of the present invention is limited to the drawings or embodiments, and it will be understood that a person skilled in the art can modify and change the present invention in various ways without departing from the spirit and scope of the present invention described in the claims below.

Claims

1. In a door image analysis-based AI (Artificial Intelligence) secretary device, an image receiving unit for receiving an image captured by a camera attached to the door, the image including an image of a person entering or exiting the door; A memory for storing the received image; and a processor that uses an artificial intelligence (AI) model to analyze the video stored in the memory to infer characteristics of the people entering and exiting the vehicle and to guide information corresponding to the inferred characteristics; The processor identifies the person entering and leaving the facility, determines whether the number of times the person entering and leaving the facility is equal to or greater than a preset reference value, and infers the characteristics of the person entering and leaving the facility only if the number of times the person is identified is equal to or greater than the reference value, thereby providing information corresponding to the inferred characteristics; The processor is connected to a motion sensor that detects a person moving; The processor, when a motion is detected within a preset range, identifies the person causing the motion and derives corresponding information by inferring characteristics of the identified person; The memory stores a plurality of pieces of memo information; The processor analyzes the text included in the memo information based on a syntax analysis algorithm to extract terms that refer to objects, and matches the video corresponding to the objects to identify whether the person entering and exiting is the object; The processor displays memo information corresponding to the person entering or exiting the door when the person enters or exits the door; The processor, when there are a plurality of people entering and exiting the facility, identifies a first person and a second person, and when the first person and the second person are identified at different times, displays memo information corresponding to the first person and the second person individually, and when the first person and the second person are identified at the same time, displays a combination of memo information corresponding to the first person and the second person. An AI secretary device based on door image analysis.

2. The door image analysis-based AI secretary device according to claim 1 , wherein the processor infers at least one of the physical condition, body shape, and body size of the person entering or exiting the door.

3. The door image analysis-based AI secretary device of claim 1 , wherein the processor recommends products corresponding to the inferred characteristics.

4. The processor infers a body type or size of the person entering or exiting the room; The door image analysis-based AI secretary device according to claim 3, which recommends corresponding clothing or products based on the inferred body type or body size.

5. the processor is in communication with an output device to provide the corresponding information via the output device; The output device is a display device for visually displaying the corresponding information; and The door image analysis-based AI secretary device according to claim 1 , further comprising a speaker for outputting the corresponding information by voice.

6. the display device is a touch screen; The door image analysis-based AI secretary device according to claim 5, wherein the memo information is displayed to the person entering or exiting through the touch screen.

7. The AI ​​secretary device communicates with a server device related to an application executed on a user terminal, The door image analysis-based AI secretary device according to claim 1 , wherein the AI ​​secretary device provides the corresponding information to the user terminal when the application is executed in the user terminal.

8. The server device communicates via a cloud-based network, a home network, or a D2D (Device to Device) system; The door image analysis-based AI secretary device according to claim 7, wherein the information stored in the memory is transmitted to the server device.

9. The processor provides advertising content to the display device for display on the display device; the display device displays the advertising content received from the processor; The door image analysis-based AI secretary device according to claim 5 , wherein the advertising content is provided by a business entity located within a certain radius of the door.

10. The processor is connected to a microphone for acquiring voice information of the person entering and exiting the vehicle; The door image analysis-based AI secretary device according to claim 1 , wherein the processor infers characteristics of the person entering or exiting the door using voice information acquired through the microphone.

11. The door image analysis-based AI secretary device of claim 1, wherein the artificial intelligence model is linked to a server that manages criminal identity information, medical information, and insurance information, and provides information related to the people entering and exiting the door using the criminal identity information, medical information, and insurance information.

12. In conjunction with an online banking server associated with the account of said person; The door image analysis-based AI secretary device according to claim 1 , further comprising: a door image analysis unit that analyzes the door image and stores the door image;

13. The processor is in communication with a wearable device attached to the person entering and exiting the facility to measure a health condition of the person entering and exiting the facility; The door image analysis-based AI secretary device according to claim 1 , which uses information acquired from the wearable device to provide medical and insurance-related information related to the person entering or exiting the door.

14. The door image analysis-based AI secretary device according to claim 13, wherein the wearable device obtains at least one of an average blood oxygen saturation level and an average heart rate wave value and provides the information to the processor.

15. The door image analysis-based AI secretary device according to claim 13, wherein the processor analyzes objects in the stored image by at least one of video, light source, and graph.

16. The door image analysis-based AI secretary device according to claim 13, wherein the data to be analyzed by the processor includes at least one of the body shape, size, time-dependent shadows, light reflection of the body by a light source, skeleton, head shape, hair, hand shape, fingerprints, palm lines, pores, blood vessels, retina, iris, face, body temperature, walking style, voice, and sound waves of the person entering or exiting the door.

17. The door image analysis-based AI secretary device according to claim 1, wherein the processor utilizes at least one of information about the person entering and exiting in an SNS server and an application server, and information stored in a memory of a device carried by the person entering and exiting, to infer characteristics of the person entering and exiting.

18. In a door image analysis-based AI (Artificial Intelligence) secretary system, A camera attached to the door to record people entering and exiting the building, An AI secretary device that uses an artificial intelligence model to analyze the image captured by the camera to infer characteristics of the people entering and leaving the facility and provides information corresponding to the inferred characteristics to an output device; and An output device that outputs information provided by the AI ​​secretary device; The AI ​​secretary device includes: Identifying the person entering and leaving the facility, and determining whether the number of times the person entering and leaving the facility has been identified is equal to or greater than a preset reference value; Only when the number of times of identification is equal to or greater than the reference value, inferring the characteristics of the person entering and leaving the facility, and providing information corresponding to the inferred characteristics; The AI ​​secretary device is connected to a motion sensor that detects human movement, When a motion is detected within a preset range, the AI ​​secretary device identifies the person causing the motion and infers characteristics of the identified person to provide corresponding information; The AI ​​secretary device stores a plurality of pieces of memo information, The AI ​​secretary device analyzes the text included in the memo information based on a syntax analysis algorithm to extract terms that refer to objects, matches the video corresponding to the object, and identifies whether the person entering or leaving is the object; The AI ​​secretary device displays memo information corresponding to the person entering or exiting the door when the person enters or exits the door; The AI ​​secretary device, when there are a plurality of people entering and exiting the facility, identifies a first person and a second person, and when the first person and the second person are identified at different times, displays memo information corresponding to the first person and the second person individually, and when the first person and the second person are identified at the same time, displays memo information corresponding to the first person and the second person in combination. An AI secretary system based on door image analysis.

Images