Electronic apparatus capable of recommending social event and operation method thereof

Abstract

Provided are an electronic apparatus capable of recommending a social event and an operation method thereof. The electronic apparatus can acquire person information, event information, and information about shooting date and time and a shooting location from a plurality of images stored in a storage, create, on the basis of at least one among the person information, the event information, and the information about the shooting date and time and the shooting location, a social event related to a particular person among people around a user, and provide a notification content for recommending the social event.

Description

Electronic device for recommending social events and method of operation thereof

[0001] The present disclosure relates to an electronic device for recommending social events and a method of operating the same. Specifically, the present disclosure discloses an electronic device and a method of operating the same that generates social events specialized for a user based on information obtained from a photograph previously stored in the electronic device and provides recommendation information for recommending the generated social events.

[0002] With the recent proliferation of camera-equipped mobile devices such as smartphones, most people now take photos using these devices. Generally, thousands to tens of thousands of photos are stored on mobile devices. Mobile devices not only simply display stored photos but also provide various functions for the user. Photo applications on mobile devices (e.g., galleries) offer a "Story" feature that classifies previously stored photos based on criteria such as objects, actions, behaviors, situations, or events, and displays related photos in a slideshow format.

[0003] Conventional story functions or various features provided by current photo applications (e.g., stories) are provided through cloud-based algorithms that recognize objects (e.g., people, animals, or items, etc.) from photos, specific actions (e.g., eating, exercising, or driving, etc.), or specific events (e.g., travel, sports games, concerts, or birthdays, etc.), and display the recognized results. While conventional photo applications provide information regarding events, this is merely based on personal details of family or friends (e.g., birthdays, hobbies, likes, or dislikes, etc.) that the user has previously saved. In other words, there is a limitation in conventional technology that, in order to provide information about events based on photos, the user must first save the personal details of relevant individuals as metadata in photos featuring specific people.

[0004] One aspect of the present disclosure provides a method for an electronic device to recommend social events. The method of operation of the electronic device may include the step of obtaining person information regarding each of the people from a plurality of previously stored images. The method of operation of the electronic device may include the step of obtaining event information from a plurality of images. The method of operation of the electronic device may include the step of obtaining at least one of a shooting date and time (shooting date and time) and a shooting location from the metadata of each of the plurality of images. The method of operation of the electronic device may include the step of obtaining a social event related to at least one of the people based on at least one of the person information, event information, shooting date and time, and shooting location. The method of operation of the electronic device may include the step of providing notification content including recommendation information regarding social events.

[0005] One aspect of the present disclosure provides an electronic device for recommending social events. The electronic device may include storage for storing a plurality of images; at least one processor including a processing circuit; memory for storing one or more instructions; and a display. By executing the one or more instructions individually or collectively by at least one processor, the electronic device may obtain person information regarding each of the persons from the plurality of images already stored in the storage. By executing the one or more instructions individually or collectively by at least one processor, the electronic device may obtain event information from the plurality of images. By executing the one or more instructions individually or collectively by at least one processor, the electronic device may obtain at least one of the shooting date and time and the shooting location from the metadata of each of the plurality of images. By executing the above one or more commands individually or collectively by at least one processor, the electronic device can acquire a social event related to at least one of the persons based on at least one of person information, event information, shooting time and place, and shooting location. By executing the above one or more commands individually or collectively by at least one processor, the electronic device can provide notification content including recommendation information regarding the social event through a display.

[0006] The present disclosure provides a computer program product comprising a computer-readable storage medium. The storage medium may include instructions readable by the electronic device to perform the following operations: obtaining person information regarding each of the persons from a plurality of images stored in the electronic device; obtaining event information from the plurality of images; obtaining at least one of a shooting date and time and a shooting location from the metadata of each of the plurality of images; obtaining a social event related to at least one of the persons based on at least one of the person information, event information, shooting date and time and a shooting location; and providing notification content including recommendation information regarding the social event.

[0007] The present disclosure can be easily understood from the combination of the following detailed description and the accompanying drawings, where reference numerals denote structural elements.

[0008] FIG. 1 is a diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure, which generates a social event based on information obtained from an image and provides recommendation information regarding the social event.

[0009] FIG. 2 is a flowchart illustrating a method in which an electronic device according to one embodiment of the present disclosure generates a social event based on information obtained from an image and provides recommendation information regarding the social event.

[0010] FIG. 3 is a block diagram illustrating the components of an electronic device according to one embodiment of the present disclosure.

[0011] FIG. 4 is a diagram illustrating data input / output between a software module stored in the memory of an electronic device and a display according to one embodiment of the present disclosure.

[0012] FIG. 5 is a diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure performing person clustering.

[0013] FIG. 6 is a drawing illustrating a person database according to one embodiment of the present disclosure.

[0014] FIG. 7 is a flowchart illustrating a method in which an electronic device according to one embodiment of the present disclosure obtains information about a first person from an image and generates a social event related to the first person.

[0015] FIG. 8 is a diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure to obtain information about a first person from an image and to generate a social event related to the first person.

[0016] FIG. 9 is a flowchart illustrating a method in which an electronic device according to one embodiment of the present disclosure obtains information about a second person from an image and generates social events related to a first person and a second person.

[0017] FIG. 10 is a diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure, which obtains information related to a social event from an image and recommends a social event related to a specific person based on the obtained information.

[0018] FIG. 11 is a flowchart illustrating a method for an electronic device according to one embodiment of the present disclosure to generate a social event related to a specific person.

[0019] FIG. 12 is a drawing illustrating a social event database according to one embodiment of the present disclosure.

[0020] FIG. 13 is a diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure acquiring notification content that recommends a social event and displaying the notification content.

[0021] FIG. 14 is a diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure generating notification content that recommends a social event and displaying the notification content.

[0022] The terms used in the embodiments of this specification have been selected to be as widely used as possible, taking into account the functions of the present disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description section of the relevant embodiments. Therefore, terms used in this specification should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0023] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this specification.

[0024] Throughout this disclosure, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part," "...module," etc., as used in this specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0025] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “system configured to” may mean that the system is “capable of” in conjunction with other devices or components. For example, the phrase “processor configured to perform A, B, and C” may mean a dedicated processor for performing the said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0026] In addition, when a component is described in the present disclosure as being "connected" or "connected" to another component, it should be understood that the component may be directly connected to or directly connected to the other component, but unless otherwise specifically stated, it may also be connected or connected through another component in between.

[0027] All functions or operations described in this disclosure may be processed individually by a single processor and / or collectively by a plurality of processors. A single processor or a combination of a plurality of processors may include circuitry that performs processing, such as an Application Processor (AP), Communication Processor (CP), Graphical Processing Unit (GPU), Neural Processing Unit (NPU), Microprocessor Unit (MPU), System on Chip (SoC), Integrated Chip (IC), etc.

[0028] It should be understood that the blocks and combinations of flowcharts in the flowcharts illustrated in the present disclosure may be performed by one or more computer programs comprising computer-executable instructions. The one or more computer programs may be stored all in a single memory or may be divided and stored in multiple different memories.

[0029] In the present disclosure, "social event" refers to a social event and represents a social, social, or social event, activity, or gathering with at least one other person. For example, a social event may include a meeting, birthday, wedding, or exercise involving at least one person.

[0030] In the present disclosure, 'personalized social event' refers to an event with a specific person specific to the user of the electronic device. For example, a personalized social event may be a meeting between the user and other people, or the birthday of a person in the user's vicinity. In one embodiment of the present disclosure, a personalized social event may be obtained based on at least one of information regarding a person recognized from images stored in the electronic device, tag information of the images, and metadata.

[0031] In the present disclosure, 'metadata' refers to structured data regarding data, which is attribute information describing other data (e.g., images such as photos or videos). Metadata is used as an index to search for specific data from a large amount of data and may include information such as, for example, the location of the data (photo or video), information on the creator of the content, rights conditions, terms of use, and usage history. In one embodiment of the present disclosure, metadata may include information regarding at least one of the location (place) where the photo or video was taken, the date of shooting, and the time of shooting. However, it is not limited thereto, and in the present disclosure, metadata may include at least one classification or identification information among, for example, objects, actions, behaviors, situations, or events.

[0032] In the present disclosure, "notification content" refers to a notification window or block that provides a notification to a user. The notification content may include at least one of an image and a notification message. In one embodiment of the present disclosure, the notification content may include a representative image associated with a personalized social event and a notification message recommending a personalized social event. The notification content may be displayed through a photo application (e.g., a gallery). However, it is not limited thereto, and the notification content may be provided, for example, as a profile page, feed, or Reels of a social network service.

[0033] In the present disclosure, functions related to 'Artificial Intelligence' are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or AI-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or AI models stored in memory. Alternatively, if the one or more processors are AI-dedicated processors, the AI-dedicated processors may be designed with a hardware structure specialized for processing a specific AI model.

[0034] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform a desired characteristic (or objective) are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0035] In the present disclosure, an 'artificial intelligence model' may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values ​​and performs neural network operations through operations between the results of operations of a previous layer and the plurality of weights. The plurality of weights possessed by the plurality of neural network layers may be optimized by the learning results of the artificial intelligence model. For example, the plurality of weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network model may include a Deep Neural Network (DNN), such as a Convolutional Neural Network, a Recurrent Neural Network, a Restricted Boltzmann Machine, a Deep Belief Network, a Bidirectional Recurrent Deep Neural Network, or Deep Q-Networks, but is not limited to the examples described above.

[0036] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0037] Embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0038] FIG. 1 shows an electronic device (100) according to one embodiment of the present disclosure having a plurality of images (i1, i2, i3, ... , i nThis is a diagram illustrating the operation of generating a social event (40) based on information (10, 20, 30) obtained from ) and providing recommendation information regarding the social event (40).

[0039] Referring to FIG. 1, the electronic device (100) may be a mobile device such as a smartphone, tablet PC, laptop computer, digital camera, e-book terminal, digital broadcasting terminal, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), navigation, or MP3 player. In the embodiment illustrated in FIG. 1, the electronic device (100) is illustrated as a smartphone, but is not limited thereto. In one embodiment of the present disclosure, the electronic device (100) may be a home appliance such as a TV, air conditioner, robot vacuum cleaner, or clothing care machine. In one embodiment of the present disclosure, the electronic device (100) may be implemented as a wearable device such as a smart watch, glasses-type augmented reality device (e.g., AR glasses), head-mounted device (HMD), or body-attached device (e.g., skin pad).

[0040] The electronic device (100) has a plurality of images (i1, i2, i3, ... , i n ) may be stored. Multiple images (i1, i2, i3, ... , i n ) may include still images or video. A still image may be, for example, a photograph. Multiple images (i1, i2, i3, ... , i n ) may be stored in storage (110, see FIG. 3) within the electronic device (100).

[0041] The electronic device (100) has a plurality of images (i1, i2, i3, ... , i n The electronic device (100) can recognize a person from ) and obtain person information (10) for each of the recognized people. The electronic device (100) can recognize a plurality of images (i1, i2, i3, ... , i n Person clustering can be performed to group recognized images from ) by person. The electronic device (100) can perform person clustering to obtain a person database containing image information by person.

[0042] The electronic device (100) has a plurality of images (i1, i2, i3, ... , i n The electronic device (100) can recognize an event from ) and obtain event information (20). The electronic device (100) can obtain person information (10) and event information (20) from a plurality of corresponding images (i1, i2, i3, ... , i n It can be saved as a tag for each.

[0043] The electronic device (100) has a plurality of images (i1, i2, i3, ... , i n ) At least one of the shooting date and time and the shooting location can be obtained from each metadata (30). In the present disclosure, the 'shooting date and time' is a plurality of images (i1, i2, i3, ... , i n ) may include information regarding each shooting date and shooting time. In one embodiment of the present disclosure, the metadata (30) may include only information regarding either the shooting date or the shooting time.

[0044] An electronic device (100) can generate social events (40) based on person information (10), event information (20), and metadata (30). In one embodiment of the present disclosure, the electronic device (100) can generate personalized social events related to at least one specific person among the people based on person information (10), event information (20), and metadata (30). In the present disclosure, 'personalized social events' refers to social events specialized for the user related to people around the user of the electronic device (100) (e.g., family, friends, colleagues, etc.). For example, personalized social events may include meetings between the user and other people, or the birthdays of people around the user.

[0045] The electronic device (100) may display notification content (50) containing recommendation information that recommends a social event (40) on a display (140). In the present disclosure, 'notification content (50)' refers to a notification window or block that provides a notification to a user. The notification content (50) may be provided, for example, as a profile page, feed, or Reels of a social network service.

[0046] In one embodiment of the present disclosure, the notification content (50) may include a representative image (52) associated with a personalized social event (40) and a notification message (54) recommending the personalized social event (40).

[0047] FIG. 2 is a flowchart illustrating a method in which an electronic device (100) according to one embodiment of the present disclosure generates a social event based on information obtained from an image and provides recommendation information regarding the social event. Hereinafter, the function and / or operation of the electronic device (100) will be described in detail with reference to FIG. 2 and FIG. 1.

[0048] In step S210, the electronic device (100) obtains person information regarding each person from a plurality of previously stored images. Referring together to FIG. 1, the electronic device (100) uses an object detection model to obtain a plurality of images (i1, i2, i3, ..., i n The electronic device (100) can recognize a person's face from the device and recognize the person based on the recognized face. The electronic device (100) can analyze the similarity between the recognized people and perform person clustering by grouping similar people together based on the similarity. The electronic device (100) can obtain a person database containing image information for each person according to the result of the person clustering. In one embodiment of the present disclosure, the person database may include information regarding at least one of person identification information, the file name of at least one image grouped by person, the storage location of at least one image, the location coordinate value of a bounding box where the person's face is detected, and event information recognized from at least one image.

[0049] In step S220 of FIG. 2, the electronic device (100) obtains event information from a plurality of images. Referring together with FIG. 1, the electronic device (100) can extract person-specific identification information from a person database. The electronic device (100) uses an event classifier to obtain event information from a plurality of images (i1, i2, i3, ..., i n Event information (20) can be obtained from ). In one embodiment of the present disclosure, the electronic device (100) obtains a plurality of images (i1, i2, i3, ... , i) in an event classification model. n Input ) and perform inference using an event classification model to obtain multiple images (i1, i2, i3, ... , i n) can obtain a confidence value that can classify as a specific event. Based on the obtained confidence value, the electronic device (100) can obtain a plurality of images (i1, i2, i3, ... , i n An event can be recognized from ). The electronic device (100) performs image tagging to obtain person identification information and event information (20) from a plurality of corresponding images (i1, i2, i3, ... , i n It can be saved as a tag for each.

[0050] In step S230 of FIG. 2, the electronic device (100) obtains at least one of the shooting date and time (shooting date and time) and the shooting location from the metadata of each of the plurality of images. In the present disclosure, 'metadata' refers to structured data regarding data, and refers to attribute information describing other data (e.g., images such as photos or videos). Metadata may include information such as the location of the data (photo or video), content creator information, rights conditions, usage conditions, usage history, etc. In one embodiment of the present disclosure, the metadata may include at least one classification information or identification information among the object, act, behavior, situation, or event of the photo or video. However, it is not limited thereto, and the metadata may include information regarding at least one of the location (place) where the photo or video was taken, the shooting date, and the shooting time. Referring together with FIG. 1, the electronic device (100) has a plurality of images (i1, i2, i3, ..., i n Information regarding the shooting date and time and the shooting location can be extracted from each metadata (30). In the present disclosure, the 'shooting date and time' is a plurality of images (i1, i2, i3, ... , i n) may include information regarding each shooting date and shooting time. In one embodiment of the present disclosure, the metadata (30) may include only information regarding either the shooting date or the shooting time.

[0051] In step S240 of FIG. 2, the electronic device (100) generates a social event related to at least one person based on at least one of person information, event information, shooting date and time, and shooting location. The electronic device (100) may obtain person information from a person database, extract event information from tags, and obtain information regarding the date, time, and shooting location where the photo was taken from metadata. In one embodiment of the present disclosure, the electronic device (100) may execute a personalized social event recommendation algorithm to generate a specialized personalized social event related to a specific person based on the obtained person information, event information, and metadata (shooting date, time, shooting location, etc.). In one embodiment of the present disclosure, the 'personalized social event' may include, for example, a meeting with a user's friend, a birthday of a user's family or acquaintance, or a regular meeting of a group to which the user belongs (e.g., a reunion, alumni association, club, etc.).

[0052] In step S250, the electronic device (100) provides notification content including recommendation information regarding a social event. Referring together to FIG. 1, the electronic device (100) includes a display (140) and can display notification content (50) recommending a personalized social event (40) on the display (140). The notification content (50) may include a representative image (52) related to the social event (40) and a notification message (54) recommending the social event (40). In one embodiment of the present disclosure, the electronic device (100) may select a notification message corresponding to the social event (40) from a plurality of notification message templates stored in a notification message database and display the selected notification message together with a representative image related to the social event (40). In one embodiment of the present disclosure, the electronic device (100) may generate a notification message recommending the social event (40) using a generative artificial intelligence model and display the generated notification message together with a representative image related to the social event (40).

[0053] Conventional electronic devices provided photo application functions (e.g., stories) that recognized objects (e.g., people, animals, or things, etc.) from stored images, specific actions (e.g., eating, exercising, or driving, etc.), or specific events (e.g., travel, sports games, concerts, or birthdays, etc.), and displayed the recognized results in a slideshow format. While the functions of conventional photo applications (e.g., stories) provide information about events, this is merely providing relevant event information based on personal details of family or friends (e.g., birthdays, hobbies, likes, or dislikes, etc.) that the user has previously saved. In other words, in order to provide information about events based on photos in the prior art, there was a limitation in that the user had to save the personal details of the relevant individuals as metadata in the photos featuring specific people.

[0054] The present disclosure aims to provide an electronic device (100) that acquires a social event specialized for a user based on information obtained from a previously stored image and displays notification content recommending the social event, and a method of operation thereof.

[0055] In the embodiment illustrated in FIGS. 1 and 2, the electronic device (100) has a plurality of previously stored images (i1, i2, i3, ... , i) even if the user does not input separate information. nIt can generate personalized social events related to people in the vicinity from the electronic device (100) and provides a technical effect of displaying a notification message (54) recommending personalized social events along with a representative image (52), such as a gathering photo or an event photo. Since the electronic device (100) according to one embodiment of the present disclosure does not require separate information input by the user, it can automatically recommend various social events that the user may have forgotten or provide them as notification content (50), thereby improving user convenience. In addition, the electronic device (100) according to one embodiment of the present disclosure not only simply displays photos or videos through a photo application (e.g., a gallery application), but also displays notification content (50) recommending social events specialized for the user, thereby increasing the frequency and duration of use of the photo application and providing a technical effect of improving the user experience of the photo application.

[0056] FIG. 3 is a block diagram illustrating the components of an electronic device (100) according to one embodiment of the present disclosure.

[0057] Referring to FIG. 3, the electronic device (100) may include storage (110), a processor (120), memory (130), and a display (140). The storage (110), processor (120), memory (130), and display (150) may each be electrically and / or physically connected to each other. FIG. 3 illustrates only essential components for explaining the function and / or operation of the electronic device (100), and the components included in the electronic device (100) are not limited to those illustrated in FIG. 3. In one embodiment of the present disclosure, when the electronic device (100) is implemented as a mobile device such as a smartphone, the electronic device (100) may further include a battery that supplies driving power to the processor (120) and the display (140). In one embodiment of the present disclosure, the electronic device (100) may further include a camera configured to capture an object and acquire an image.

[0058] Storage (110) is a local storage space within an electronic device (100) that stores multiple images, such as photos or videos. Multiple images may be stored in the storage (110). When an image is acquired by a camera of the electronic device (100), the acquired image may be stored in the storage (110).

[0059] Storage (110) may be implemented as non-volatile memory. 'Non-volatile memory' refers to a storage medium that stores and maintains information even when power is not supplied, and can use the stored information again when power is supplied. Non-volatile memory may include, for example, at least one of flash memory, hard disk, SSD (Solid State Drive), multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), ROM (Read Only Memory), magnetic memory, magnetic disk, and optical disk.

[0060] In FIG. 3, the storage (110) is depicted as a component included within the electronic device (100) and as a component separate from the memory (130), but is not limited thereto. In one embodiment of the present disclosure, the storage (110) may be included within the memory (130). However, it is not limited thereto, and in one embodiment of the present disclosure, the storage (110) may not be included as a local component in the electronic device (100) but may be implemented as web storage or a cloud server. In this case, the electronic device (100) further includes a communication interface configured to perform data transmission and reception with an external web storage or cloud server, and through the communication interface, access a plurality of images stored in the web storage or cloud server and receive information about the plurality of images.

[0061] The processor (120) can execute one or more instructions of a program stored in memory (130). The processor (120) may be composed of hardware components that perform arithmetic, logic, and input / output operations and image processing. Although the processor (120) is depicted as a single element in FIG. 3, it is not limited thereto. In one embodiment of the present disclosure, the processor (120) may be composed of one or more elements.

[0062] The processor (120) may include various processing circuits and / or multiple processors. For example, the term "processor" as used in the present disclosure, including in the claims, may include at least one processor and various processing circuits. In the at least one processor, one or more processors may be configured to perform the various functions described herein in a distributed manner, individually and / or collectively. As used in the present disclosure, "processor," "at least one processor," and "one or more processors" may be configured to perform various functions. However, these terms cover, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, the at least one processor may include a combination of processors performing various functions of the disclosed functions in a distributed manner. The at least one processor may execute program instructions to achieve or perform various functions.

[0063] The processor (120) may be implemented as a general-purpose processor such as a CPU (Central Processing Unit), AP (Application Processor), DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU (Graphic Processing Unit) or VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU (Neural Processing Unit). The processor (120) may be controlled to process input data according to predefined operation rules or an artificial intelligence model. Alternatively, if the processor (120) is an artificial intelligence-dedicated processor, the artificial intelligence-dedicated processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0064] The memory (130) may be composed of at least one type of storage medium, such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), or an optical disk.

[0065] The memory (130) may store instructions related to functions and / or operations in which an electronic device (100) obtains information from a plurality of images stored in storage (110), generates social events based on the obtained information, and displays notification content recommending the generated social events. In one embodiment of the present disclosure, the memory (130) may store at least one of instructions, an algorithm, a data structure, program code, and an application program that can be read by a processor (120). The instructions, algorithm, data structure, and program code stored in the memory (130) may be implemented in a programming or scripting language such as, for example, C, C++, Java, assembler, etc.

[0066] The memory (130) may store instructions, algorithms, data structures, or program code regarding a person clustering module (131), an event recognition module (132), a metadata extraction module (133), a social event generation module (134), and a notification content generation model (135). A 'module' included in the memory (130) refers to a unit that processes a function or operation performed by the processor (120), and this can be implemented as software such as instructions, algorithms, data structures, or program code.

[0067] The processor (120) can be implemented by executing instructions or program codes stored in memory (130). Hereinafter, with reference to FIGS. 3 and FIGS. 4 together, the functions and / or operations performed by the processor (120) by executing instructions or program codes of each of the plurality of modules stored in memory (130), and the data input / output between the plurality of modules (person clustering module (131), event recognition module (132), metadata extraction module (133), social event generation module (134), and notification content generation module (135)) and components (e.g., storage (110), and display (140)) will be described in detail.

[0068] FIG. 4 is a diagram illustrating data input / output between a software module stored in a memory (130) of an electronic device (100) according to one embodiment of the present disclosure and a display. Although not shown in FIG. 4, a processor (120, see FIG. 3) may execute instructions or program code of a person clustering module (131), an event recognition module (132), a metadata extraction module (133), a social event generation module (134), and a notification content generation module (135) to perform related functions and / or operations.

[0069] Referring to FIG. 3 and FIG. 4 together, the processor (120) executes instructions or program code of the person clustering module (131), the event recognition module (132), and the metadata extraction module (133) to access the storage (110), thereby accessing a plurality of images (i1, i2, i3, ... , i) stored in the storage (110). n Information can be obtained from ).

[0070] The person clustering module (131) is a plurality of images (i1, i2, i3, ... , i) already stored in storage (110). nIt consists of instructions or program code for executing a function and / or operation of recognizing people from ), grouping by person based on the recognition result, and obtaining a person database containing image information for each grouped person. The processor (120) executes the instructions or program code of the person clustering module (131) to obtain a plurality of images (i1, i2, i3, ... , i n The processor (120) can recognize figures from ) and obtain figure information for each of the recognized figures. The processor (120) can recognize figures from a plurality of images (i1, i2, i3, ... , i n The processor (120) can recognize the faces of people from the processor and recognize people based on the recognized faces. The processor (120) can analyze the similarity between the recognized people and perform person clustering by grouping similar people together based on the similarity. The processor (120) can obtain a person database containing image information for each person based on the results of the person clustering.

[0071] In one embodiment of the present disclosure, a person clustering module (131) may include a face recognition model (510, see FIG. 5), a feature extraction model (520, see FIG. 5), and a clustering model (530, see FIG. 5). A processor (120) uses the models included in the person clustering module (131) to perform a plurality of images (i1, i2, i3, ..., i n A specific embodiment of recognizing a person's face from ) and clustering by person based on the recognition result will be described in detail in FIG. 5.

[0072] In one embodiment of the present disclosure, a person database may include information regarding at least one of person identification information, the file name of at least one image grouped by person, the storage location of at least one image, the location coordinates of a bounding box where a person's face is detected, and event information recognized from at least one image. The person database will be described in detail with reference to FIG. 6.

[0073] The event recognition module (132) has a plurality of images (i1, i2, i3, ... , i nIt consists of commands or program code for executing a function and / or operation to recognize an event from ). In one embodiment of the present disclosure, the event recognition module (132) may include an event classifier trained to output a confidence value indicating the probability that the input image contains a predefined event when an image is input. The event classifier may be composed of a deep neural network model trained through a supervised learning method that applies tens of thousands or hundreds of millions of training images as input and applies a label value regarding an event included in each of the images as the ground truth. Here, the input image is a 640×640×3 image, and the ground truth may be a label value containing 1×N feature values. Here, N may be the number of predefined event tags. The deep neural network model may be implemented, for example, as a Convolutional Neural Network. The event classifier can be implemented, for example, using Mobilenet v4. However, it is not limited to this, and the deep neural network model may also be implemented using, for example, a Recurrent Neural Network, a Restricted Boltzmann Machine, a Deep Belief Network, a Bidirectional Recurrent Deep Neural Network, Deep Q-Networks, or a Transformer.

[0074] The processor (120) handles a plurality of images (i1, i2, i3, ..., i) stored in the storage (110). n) is applied as input to the event classifier of the event recognition module (132), and by performing inference using the event classifier, a plurality of images (i1, i2, i3, ... , i n It can recognize events from ) and obtain event information regarding the recognized events.

[0075] The metadata extraction module (133) is a plurality of images (i1, i2, i3, ... , i n It consists of instructions or program code for executing a function and / or operation to extract metadata from ). In the present disclosure, 'metadata' refers to structured data regarding data, and refers to attribute information describing other data (e.g., images such as photos or videos). Metadata may include information such as, for example, the location of the data (photo or video), content creator information, rights conditions, usage conditions, usage history, etc. In one embodiment of the present disclosure, metadata may include at least one classification information or identification information among the object, act, behavior, situation, or event of the photo or video. However, it is not limited thereto, and metadata may include information regarding at least one of the location (place) where the photo or video was taken, the date of shooting, and the time of shooting. The processor (120) executes the instructions or program code of the metadata extraction module (133) to extract a plurality of images (i1, i2, i3, ..., i n Information regarding at least one of the shooting date and time and the shooting location can be extracted from each of the metadata. In the present disclosure, the 'shooting date and time' refers to a plurality of images (i1, i2, i3, ... , i n ) It may include information regarding each shooting date and shooting time. In one embodiment of the present disclosure, the metadata may include only information regarding either the shooting date or the shooting time.

[0076] The social event generation module (134) is composed of commands or program code for executing a function and / or operation to generate a social event based on input information. In one embodiment of the present disclosure, the social event generation module (134) may execute a function and / or operation to generate a personalized social event related to a specific person among the user's surrounding people based on input information. In the present disclosure, a 'personalized social event' refers to an event with a specific person specialized for the user of the electronic device (100). For example, a personalized social event may be a meeting between the user and other people, or the birthday of a person surrounding the user. Referring to FIG. 4, a person database and person information obtained by the person clustering module (131), event information obtained by the event recognition module (132), and metadata extracted by the metadata extraction module (133) (e.g., shooting date, time, shooting location, etc.) may be input to the social event generation module (134). The processor (120) can generate social events specialized for a user in relation to a specific person based on a person database, person information, event information, and metadata by executing commands or program code of the social event generation module (134). The social events generated by the processor (120) may include, for example, a meeting with the user's friends, the birthday of the user's family or acquaintances, or a regular meeting of a group to which the user belongs (e.g., a reunion, alumni association, club, etc.). The processor (120) can input information regarding the personalized social events generated by the social event generation module (134) into the notification content generation module (135).

[0077] The notification content generation module (135) is composed of instructions or program code for executing a function and / or operation to generate notification content including recommendation information regarding social events. The processor (120) can generate notification content recommending personalized social events by executing the instructions or program code of the notification content generation module (135). In the present disclosure, 'notification content' refers to a window or block that provides a recommendation or notification regarding a personalized social event to a user. In one embodiment of the present disclosure, the notification content may include a representative image related to the personalized social event and a notification message recommending the personalized social event.

[0078] In one embodiment of the present disclosure, the processor (120) may select a notification message corresponding to a personalized social event from a plurality of notification message templates stored in a notification message database, and generate notification content using the selected notification message. A specific embodiment in which the processor (120) generates notification content using a plurality of notification message templates stored in a notification message database will be described in detail with reference to FIG. 13.

[0079] In one embodiment of the present disclosure, the processor (120) may generate a notification message recommending a personalized social event using a generative artificial intelligence model, and may generate notification content using the generated notification message. The generative artificial intelligence model may be implemented as an artificial intelligence model trained to generate a response message to an input prompt. Specific embodiments of how the processor (120) generates notification content using a generative artificial intelligence model will be described in detail with reference to FIG. 14.

[0080] The display (140) is configured to display notification content under the control of the processor (120). In one embodiment of the present disclosure, the display (140) may display images through a photo application (e.g., a gallery application) and display notification content through a personalized notification window of the photo application.

[0081] The display (140) can be implemented as at least one of, for example, a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a 3D display, and an electrophoretic display.

[0082] FIG. 5 is a diagram illustrating the operation of an electronic device (100) according to one embodiment of the present disclosure performing person clustering.

[0083] Referring to FIG. 5, the person clustering module (131) may include a face recognition model (510), a feature extraction model (520), and a clustering model (530). The processor (120) can recognize people from an image and group the image by person by executing instructions or program code of the models included in the person clustering module (131).

[0084] The face recognition model (face detector) (510) may be composed of an artificial intelligence model trained to recognize a face from an image and detect a bounding box of the recognized face portion. In one embodiment of the present disclosure, the face recognition model (510) may be composed of a deep neural network model trained through a supervised learning method that applies numerous images composed of training data as input and applies the location coordinates of the bounding box of the face portion included in each of the images as ground truth. For example, the face recognition model (510) may be trained to output the location coordinates of the bounding box of the face portion (e.g., [x_center, y_center, width, height]) when a 640×640×3 image is input. The deep neural network model may be implemented, for example, as a Convolutional Neural Network. The face recognition model (510) can be implemented, for example, as a YOLO model. However, it is not limited thereto, and the deep neural network model can be implemented, for example, as a recurrent neural network, a restricted Boltzmann machine, a deep belief network, a bidirectional recurrent deep neural network, a deep Q-network, or a transformer.

[0085] The processor (120) inputs an image into a face recognition model (510) included in a person clustering module (131), performs inference using the face recognition model (510) to recognize a face part from the image, and can obtain location coordinate values ​​of a bounding box of the recognized face part. The processor (120) can input an image of a bounding box specified by location coordinate values ​​into a feature extraction model (520). The image of the bounding box may be, for example, an image having a size of 30×30×3.

[0086] The face feature extractor (520) may be composed of an artificial intelligence model trained to extract face features from an input face image. In one embodiment of the present disclosure, the face feature extractor (520) may be composed of a deep neural network model trained through a supervised learning method that applies numerous training face images as input and applies an n-dimensional feature vector of each face image as the correct answer. For example, the face feature extractor (520) may be trained to output a 256-dimensional face feature vector (e.g., 1×256 feature, float 32) when a 30×30×3 image is input. The deep neural network model may be implemented, for example, as a Convolutional Neural Network. The face feature extractor (520) may be implemented, for example, as a Mobilenet v4 model. However, this is not limited to this, and deep neural network models may also be implemented as, for example, recurrent neural networks, restricted Boltzmann machines, deep belief networks, bidirectional recurrent deep neural networks, deep Q-networks, or transformers.

[0087] The processor (120) can perform inference using a feature extraction model (520) to extract a 256-dimensional feature vector from a bounding box image input from a face recognition model (510). The processor (120) can input the extracted 256-dimensional feature vector into a clustering model (530).

[0088] The clustering model (face cluster) (530) may be composed of an artificial intelligence model trained to perform person clustering by grouping input feature vectors by person. In one embodiment of the present disclosure, the clustering model (530) may be composed of a Graph Convolutional Network (GCN) composed of a graph convolution layer that applies a face feature vector as input, analyzes the input feature vector on an n-dimensional graph, and performs clustering based on the analysis results of the graph. For example, the clustering model (530) may be trained to output a confidence value representing the cluster result when a 256-dimensional feature vector (e.g., 1×256 feature, float 32) is input. Here, the confidence value has a size of (N, M), where N represents the number of feature vectors and M represents the cluster ID.

[0089] The processor (120) inputs a 256-dimensional face feature vector output by the feature extraction model (520) into a clustering model (530) and can obtain a confidence value by clustering by face using the clustering model (530). The processor (120) can group images by person using the confidence value and obtain a person database containing image information for each grouped person. The person database will be described in detail with reference to FIG. 6.

[0090] FIG. 6 is a drawing illustrating a person database (600) according to one embodiment of the present disclosure.

[0091] Referring to FIG. 6, the person database (600) may be a set of data that stores information of images grouped into people according to person identification information (610). The person database (600) may include data regarding person identification information (610), filename (620), storage location (630), face coordinates (640), and events (650).

[0092] Person identification information (610) is information for identifying a person grouped by a person clustering module (131, see FIG. 5). In the embodiment illustrated in FIG. 6, the person identification information (610) is illustrated with serial numbers such as 001, 002, ..., but is not limited thereto. In one embodiment of the present disclosure, the person identification information (610) may be composed of English letters or a combination of English letters and numbers as well as numbers. In one embodiment of the present disclosure, when a specific name is assigned to a person, for example, when person 001 is assigned 'Father' and person 002 is assigned 'Co-worker', the person identification information (610) may include information assigned by the user (e.g., 'Father', 'Colleague').

[0093] The filename (620) may include information regarding the name of a file in which a specific person is recognized among multiple images. For example, the name of at least one file in which 'person 001' is recognized may be abcde.jpg, xxxxx.jpg, and yyyyy.jpg, and the name of at least one file in which 'person 002' is recognized may be zzzzz.jpg and 11111.jpg.

[0094] The storage location (630) may include information regarding the storage location of a file in which a specific person is recognized. In one embodiment of the present disclosure, the storage location (630) may be a folder name in the storage within the electronic device (100). For example, abcde.jpg, a file grouped by person 001, may be stored in the 'Family' folder within the storage, and zzzzz.jpg, a file grouped by person 002, may be stored in the 'Work Colleagues' folder within the storage.

[0095] The face coordinates (640) may include position coordinate values ​​of the bounding box of the face portion detected from the image. In one embodiment of the present disclosure, the face coordinates (640) may include information regarding the width and height as well as position coordinate values ​​of the X-axis and Y-axis of the bounding box. For example, the coordinate values ​​of the face detected in abcde.jpg, a file grouped as person 001, may be (x1, y1, w1, h1), and the coordinate values ​​of the face detected in zzzzz.jpg, a file grouped as person 002, may be (x4, y4, w4, h4).

[0096] The event (650) may include information about the event recognized from the image. For example, the event recognized from abcde.jpg, a file grouped by person 001, may be 'meeting', and the event recognized from zzzzz.jpg, a file grouped by person 002, may be 'exercise'.

[0097] FIG. 7 is a flowchart illustrating a method in which an electronic device (100) according to one embodiment of the present disclosure obtains information about a first person from an image and generates a social event related to the first person.

[0098] FIG. 8 is a diagram illustrating the operation of an electronic device (100) according to one embodiment of the present disclosure, which obtains information about a first person from an image and generates a personalized social event (820) related to the first person.

[0099] Hereinafter, the function and / or operation of the electronic device (100) will be described in detail with reference to FIG. 7 and FIG. 8 together.

[0100] Referring to FIG. 7, step S710 is a step that embodies the operation of step S210 illustrated in FIG. 2. In step S710, the electronic device (100) obtains a first person database containing information regarding at least one image grouped as a first person among the people recognized from a plurality of images. In one embodiment of the present disclosure, the processor (120, see FIG. 3) of the electronic device (100) may perform person clustering to recognize the face of a person from each of the plurality of images, obtain a feature vector from the recognized face, and group the plurality of images by person by clustering by face based on the feature vector. The processor (120) may obtain a person database containing image information for each grouped person.

[0101] Referring together to the embodiment illustrated in FIG. 8, the processor (120) may perform person clustering to obtain a person database (800) containing image information regarding a first person (friend 1) and a second person (friend 2). The first person database (801) may store information regarding identification information of the first person (friend 1), at least one image in which the first person (friend 1) is recognized, the file name of at least one image, the storage location of at least one image, and at least one bounding box in which the face of the first person (friend 1) is recognized within at least one image. The second person database (802) may store information regarding identification information of the second person (friend 2), at least one image in which the second person (friend 2) is recognized, the file name of at least one image, the storage location of at least one image, and at least one bounding box in which the face of the second person (friend 2) is recognized within at least one image.

[0102] Referring again to FIG. 7, step S720 is a step that embodies the operation of step S230 illustrated in FIG. 2. In step S720, the electronic device (100) obtains information regarding the most recent shooting date from the metadata of at least one image included in the first person database. In one embodiment of the present disclosure, the processor (120) of the electronic device (100) may extract information regarding the shooting date or shooting time from the metadata of at least one image included in the person database associated with a specific person. Referring together to the embodiment illustrated in FIG. 8, the processor (120) may obtain information regarding the shooting date on which at least one image was taken from the metadata (811) of at least one image included in the first person database (801). The processor (120) may extract information regarding the shooting date from the metadata of the most recently taken and updated image among the at least one images. However, it is not limited to this, and the processor (120) may further obtain information regarding the shooting time and shooting location where at least one image was taken from the metadata (811) of at least one image included in the first person database (801).

[0103] Steps S730 to S770 of FIG. 7 are steps that embody the operation of step S240 illustrated in FIG. 2. In step S730, the electronic device (100) counts the number of at least one image included in the first person database and can determine whether the number of at least one image is greater than or equal to a preset first threshold (α). The first threshold (α) is a value that serves as a criterion for determining whether to perform a social event with the first person, and may be a value preset regarding the number of images included in the person database. For example, the first threshold (α) may be 50, but is not limited thereto.

[0104] When the number of at least one image included in the first person database is compared with a first threshold (α) and the number of at least one image is greater than or equal to the first threshold (α) (step S740), the electronic device (100) compares the number of days between the most recent shooting date of at least one image and the current time point with a preset second threshold (β) to determine whether the number of days is greater than or equal to the second threshold (β). In one embodiment of the present disclosure, the electronic device (100) can calculate the number of days regarding how long ago the most recent shooting date was based on the current time point, based on information regarding the most recent shooting date obtained in step S720. The electronic device (100) can compare the calculated number of days with the second threshold (β). The second threshold (β) is a value that serves as a criterion for determining how long ago the meeting with the first person was from the current time point, and may be a preset value. For example, the second threshold (β) may be one year, but is not limited thereto.

[0105] When comparing the number of days between the most recent shooting date and the current time with a second threshold (β), if the number of days is greater than or equal to the second threshold (β) (step S750), the electronic device (100) determines that there has been no recent meeting with the first person. For example, if the number of at least one image included in the first person database is 50 or more, and the most recent shooting date extracted from the metadata of at least one image is one year or more from the current time, the processor (120) may determine that there has been no recent meeting with the first person.

[0106] In step S770, the electronic device (100) generates a social event that recommends a meeting with the first person.

[0107] After the operation of step S770 is performed, the operation of step S250 illustrated in FIG. 2 may be performed. In step S250, the electronic device (100) may display notification content recommending a meeting with the first person. Referring together to the embodiment illustrated in FIG. 8, the electronic device (100) may generate a personalized social event (820) regarding a meeting with the first person on the display (140) and display notification content (830) recommending a meeting with the first person. The notification content (830) may include a representative image (831) regarding the first person and a notification message (832) recommending a meeting with the first person. The representative image (831) may be selected from at least one image included in the first person database (801). In the embodiment illustrated in FIG. 8, the representative image (831) is shown as two images, but is not limited thereto. The representative image (831) may be one or more than two. The notification message (832) may be a message recommending a meeting or contact with the first person (Friend 1), such as, "It’s a friend you haven’t been in touch with lately. How about reaching out for the first time in a while?"

[0108] In step S730 of FIG. 7, if the number of at least one image included in the first person database is less than the first threshold (α) (step S780), the process terminates without generating a social event with the first person. In this case, the processor (120) may determine that a social event with the first person is unnecessary.

[0109] In step S740, if the number of days between the most recent shooting date and the present time is less than the second threshold (β) (step S760), the electronic device (100) determines that there has been a recent meeting with the first person. In this case, the electronic device (100) terminates without generating a social event with the first person (step S780).

[0110] FIG. 9 is a flowchart illustrating a method in which an electronic device (100) according to one embodiment of the present disclosure obtains information about a second person from an image and generates social events related to a first person and a second person.

[0111] Steps S910 to S970 illustrated in FIG. 9 are steps that embody the operation according to step S240 of FIG. 2. After the operation of step S970 illustrated in FIG. 9 is performed, the operation of step S250 of FIG. 2 may be performed.

[0112] Step S910 may be performed after the operation of Step S740 illustrated in FIG. 7 has been performed. In Step S910, the electronic device (100) identifies an image in which the first person and the second person are recognized together among at least one image. In one embodiment of the present disclosure, the processor (120, see FIG. 3) of the electronic device (100) may identify at least one image in which the first person and the second person are recognized together among at least one image included in the first person database (801, see FIG. 8). The processor (120) may access the second person database (802, see FIG. 8) to identify at least one image in which the second person is recognized, and may use information of the identified at least one image (e.g., filename, storage location, etc.) to identify at least one image in which the first person as well as the second person are recognized together among at least one image included in the first person database (801).

[0113] In step S920, the electronic device (100) counts the number of at least one identified image and can determine whether the counted number of at least one image is greater than or equal to a preset third threshold (γ). The third threshold (γ) is a value that serves as a criterion for determining whether to perform a social event with the first person and the second person, and may be a preset value regarding the number of images included in the person database. The third threshold (γ) may be equal to or smaller than the first threshold (α). The third threshold (γ) may be, for example, 10, but is not limited thereto.

[0114] When the number of at least one identified image is compared with a third threshold (γ) and the number of at least one image is greater than or equal to the third threshold (γ) (step S930), the electronic device (100) obtains information regarding the most recent shooting date from the metadata of the identified image. In one embodiment of the present disclosure, the processor (120) may obtain information regarding the shooting date on which at least one image was taken from the metadata of at least one image in which both the first person and the second person are recognized among at least one image included in the first person database (801). The processor (120) may extract information regarding the shooting date from the metadata of the most recently taken and updated image among at least one image. Referring together with the embodiment illustrated in FIG. 8, the processor (120) can extract information regarding the shooting date of at least one image in which the first person is recognized from the metadata (811) of at least one image included in the first person database (801), and extract information regarding the shooting date of at least one image in which the second person is recognized from the metadata (812) of at least one image included in the second person database (802). The processor (120) can obtain information regarding the most recent shooting date of at least one image in which both the first person and the second person are recognized among the information regarding the shooting date.

[0115] In step S940, the electronic device (100) compares the number of days between the most recent shooting date of at least one image and the current time point with a preset second threshold (β) to determine whether the number of days is greater than or equal to the second threshold (β). In one embodiment of the present disclosure, the electronic device (100) may calculate the number of days regarding how long ago the most recent shooting date was relative to the current time point, based on information regarding the most recent shooting date obtained in step S930. The electronic device (100) may compare the calculated number of days with the second threshold (β). The second threshold (β) may be a preset value that serves as a criterion for determining how long ago the meeting with the first person and the second person was from the current time point. For example, the second threshold (β) may be one year, but is not limited thereto.

[0116] When comparing the number of days between the most recent shooting date and the current time with a second threshold (β), if the number of days is greater than or equal to the second threshold (β) (step S950), the electronic device (100) determines that there has been no recent meeting with the first person and the second person. For example, if the number of images in which the first person and the second person are recognized together is 10 or more, and the most recent shooting date extracted from metadata is 1 year or more from the current time, the processor (120) may determine that there has been no recent meeting with the first person and the second person.

[0117] In step S970, the electronic device (100) generates a social event that recommends a meeting with the first person and the second person.

[0118] After the operation of step S970 is performed, the operation of step S250 illustrated in FIG. 2 may be performed. In step S250, the electronic device (100) may display notification content recommending a meeting with the first person and the second person. Referring together to the embodiment illustrated in FIG. 8, the electronic device (100) may generate a personalized social event (820) regarding a meeting with the first person and the second person on the display (140) and display notification content (840) recommending a meeting with the first person and the second person. The notification content (840) may include a representative image (841) among at least one image in which the first person and the second person are recognized together, and a notification message (842) recommending a meeting with the first person and the second person. The representative image (841) may be selected from at least one image in which both the first person and the second person are recognized among the images included in the first person database (801). In the embodiment illustrated in FIG. 8, the representative image (841) is shown as two images, but is not limited thereto. The representative image (841) may be one or more than two images. The notification message (842) may be a text message recommending a meeting or contact with a first person (Friend 1) and a second person (Friend 2), such as, for example, "It’s been a long time since I’ve seen my friends. How about making plans for the first time in a while?"

[0119] FIG. 9 relates to an embodiment that, unlike the embodiment illustrated in FIG. 7, generates a personalized social event such as a meeting with multiple people (e.g., 'first person (friend 1) and second person (friend 2)') rather than a meeting (personalized social event) related to a single person (e.g., 'first person (friend 1)'), and displays notification content (840, see FIG. 8) recommending the personalized social event. Although the embodiment and notification content (840) of FIG. 9 are illustrated and described only for a personalized social event related to the first person and the second person, the present disclosure is not limited thereto. In one embodiment of the present disclosure, the electronic device (100) may generate a personalized social event related to three or more people belonging to a group to which the user belongs, e.g., a reunion, an alumni association, a club, or a sports club, and display notification content recommending the personalized social event.

[0120] When comparing the number of days between the most recent shooting date and the current time with a second threshold (β), if the number of days is less than the second threshold (β) (step S960), the electronic device (100) determines that there was a recent meeting with the first person and the second person. For example, if the most recent shooting date extracted from the metadata of an image in which the first person and the second person are recognized together is three months prior to the current time, the processor (120) may determine that there was a recent meeting with the first person and the second person.

[0121] In step S980, the electronic device (100) terminates without generating a social event with the first person and the second person.

[0122] If, as a result of the judgment in step S920, the number of at least one identified image is less than the third threshold (γ) (step S980), the electronic device (100) terminates without generating a social event with the first person and the second person. In this case, the electronic device (100) may determine that a social event (e.g., a meeting) with the first person and the second person is not necessary.

[0123] FIG. 10 is a diagram illustrating the operation of an electronic device (100) according to one embodiment of the present disclosure, which obtains information related to a social event from an image (i) and recommends a personalized social event related to a specific person based on the obtained information.

[0124] Referring to FIG. 10, the electronic device (100) can identify an image (i) related to a social event among a plurality of images already stored in storage (110, see FIG. 3) based on tag information. The social event may be, for example, a birthday. In the embodiment illustrated in FIG. 10, the processor (120, see FIG. 3) of the electronic device (100) can identify an image (i) among a plurality of images that has '#birthday' in its tag.

[0125] The processor (120) can recognize people from an image (i) and group the recognized people by person by executing instructions or program code of the person clustering module (131). The processor (120) can obtain a person database containing image information by person based on the grouping result.

[0126] The processor (120) can recognize an event from an image (i) by executing instructions or program code of the event recognition module (132).

[0127] The processor (120) can extract information regarding the shooting date, time, and shooting location from the metadata of the image (i) by executing the instructions or program code of the metadata extraction module (133).

[0128] The specific method by which the processor (120) obtains a person database, events, and metadata from an image (i) is the same as described in FIGS. 1 to 4, so a redundant description is omitted.

[0129] The person database obtained by the person clustering module (131), the event information obtained by the event recognition module (132) (e.g., 'birthday'), and the metadata extracted by the metadata extraction module (133) (e.g., shooting date, time, shooting location, etc.) can be input into the social event generation module (134). The processor (120) can generate a personalized social event related to a specific person based on the person information, event information, and metadata by executing the commands or program code of the social event generation module (134). The personalized social event may be, for example, the birthday of a user's family or friend. A specific method for the processor (120) to generate a personalized social event related to a specific person will be described with reference to FIG. 11.

[0130] FIG. 11 is a flowchart illustrating a method in which an electronic device (100) according to one embodiment of the present disclosure generates a personalized social event related to a specific person.

[0131] Step S1110 can be performed after the operation of Step S230 illustrated in FIG. 2 has been performed. In Step S1110, the electronic device (100) identifies an image related to a social event among a plurality of images based on a tag. For example, the electronic device (100) can identify an image among a plurality of images already stored in storage (110, see FIG. 3) that has '#birthday' included in the tag.

[0132] Steps S1120, S1130, S1140, and S1160 are steps that embody the operation of step S240 illustrated in FIG. 2. In step S1120, the electronic device (100) uses an object recognition model to recognize a person related to a social event from an identified image. The object recognition model may be composed of an artificial intelligence model trained to recognize objects from an input image and output a classification result of the recognized objects. The processor (120, see FIG. 3) of the electronic device (100) inputs the image identified in step S1110 into the object recognition model and can recognize at least one object from the image using the object recognition model. For example, the processor (120) can use the object recognition model to recognize objects related to a birthday, such as a party hat, candles, and a cake, from the image.

[0133] In step S1130, the electronic device (100) can determine whether a person associated with a social event is recognized among the people included in the identified image. For example, if the processor (120) recognizes a person wearing a party hat or blowing out a candle from the image, the person may recognize the person associated with a social event, e.g., a 'birthday'.

[0134] When a person associated with a social event is recognized from an image (step S1140), the electronic device (100) generates a social event associated with the recognized person. In one embodiment of the present disclosure, the processor (120) may obtain person information from a person database and extract information regarding the shooting date from metadata to generate a personalized social event associated with the person (e.g., a birthday party).

[0135] Steps S1150 and S1170 are steps that embody the operation of step S250 shown in FIG. 2.

[0136] In step S1150, the electronic device (100) displays notification content including a representative image of a recognized person and a notification message recommending a social event. Referring together to the embodiment illustrated in FIG. 10, information of a personalized social event (e.g., 'birthday party') is input into a notification content generation module (135), and the processor (120) can generate notification content (1000) recommending a personalized social event (e.g., 'birthday party') by executing instructions or program code of the notification content generation module (135). The processor (120) can control the display (140) to display the notification content (1000). In the embodiment illustrated in FIG. 10, the notification content (1000) displayed on the display (140) may include a representative image (1002) of a second person recognized as a person associated with a social event (e.g., 'birthday') as a result of object recognition, and a notification message (1004) recommending a birthday party. The notification message (1004) may be a text message such as, for example, "Person 2's (second person's) birthday is coming up. How about preparing a gift?"

[0137] If a person associated with a social event is not recognized in step S1130 (step S1160), the electronic device (100) generates a personalized social event based on person information, tags, and metadata. In this case, the processor (120) can generate a personalized social event (e.g., 'birthday party') based on person information obtained from a person database, tag information regarding an event (e.g., 'birthday'), and metadata (e.g., shooting date), without specifying a person associated with a social event.

[0138] In step S1170, the electronic device (100) displays notification content including a notification message recommending a personalized social event. Referring together to the embodiment illustrated in FIG. 10, the processor (120) can control the display (140) to display the notification content (1010). In the embodiment illustrated in FIG. 10, the notification content (1010) displayed on the display (140) may include an entire image related to a specific person associated with a social event (e.g., 'birthday') as a representative image (1012). The notification content (1010) may include the representative image (1012) and a notification message (1014) recommending a birthday party. The notification message (1014) may be a message unrelated to the specific person. The notification message (1014) may be a text message that recommends only a personalized social event (e.g., 'birthday party') without being related to a specific person, such as, for example, "Your birthday is coming up. Try preparing a party."

[0139] In one embodiment of the present disclosure, the electronic device (100) may display notification content (1000, 1010) prior to a preset date from a date associated with a social event. For example, if the social event is a 'birthday', the processor (120) may display notification content (1000, 1010) recommending a birthday party prior to a preset date (e.g., one week) from the next year's birthday.

[0140] FIG. 12 is a drawing illustrating a social event database according to one embodiment of the present disclosure.

[0141] Referring to FIG. 12, the electronic device (100) can generate a social event database (1200) based on information obtained from an image. The processor (120, see FIG. 3) of the electronic device (100) can recognize people (1210) from an image in which a social event is identified and perform person clustering to group the recognized people (1210) by person. In one embodiment of the present disclosure, the processor (120) can detect a bounding box for each face of the recognized people (1210), extract a feature vector from the bounding box, and group by person based on the extracted feature vector. In the embodiment illustrated in FIG. 12, the processor (120) can cluster the images of the people (1210) into a first person (1210-1), a second person (1210-2), and a third person (1210-3).

[0142] The processor (120) can obtain event information (1220) from the tags of the image. The event information (1220) may include tags related to social events, such as #birthday, #birthday party, #family, or #cake.

[0143] The processor (120) can extract information regarding the shooting date (1230) from the metadata of the image. The shooting date (1230) may be, for example, October 12, 2023.

[0144] The processor (120) can create a social event database (1200) using information regarding grouped people (1210-1, 1210-2, 1210-3), tag event information (1220), and shooting date (1230). The processor (120) can create personalized social events using information included in the social event database (1200).

[0145] The electronic device (100) can recognize a person related to a social event from a social event database (1200) and generate a personalized social event database (1240, 1250) based on the recognition result. In one embodiment of the present disclosure, the processor (120) of the electronic device (100) can recognize an object from an image using an object recognition model and determine a person related to a social event based on the recognized object. For example, if the processor (120) recognizes a party hat, a candle, or a cake, etc. from an image, the person wearing the party hat or blowing out the candle can be recognized as a person related to a social event, for example, a 'birthday'. In the embodiment illustrated in FIG. 12, the processor (120) can determine a second person (1210-2) related to a birthday among the people (1210-1, 1210-2, 1210-3) recognized from the image. When a second person (1210-2) associated with a birthday is determined, the processor (120) may generate a personalized social event database (1240) associated with the second person (1210-2). The personalized social event database (1240) may include information on a personalized social event (e.g., 'birthday') associated with the second person (1210-2). In one embodiment of the present disclosure, the personalized social event database (1240) may include information regarding a face image of the second person (1210-2) and the date (1242) of the image being taken.

[0146] If, as a result of recognizing an object from an image using an object recognition model, a person related to a social event is not determined, the processor (120) may not specify the person related to the social event and may generate a personalized social event database (1250) based on person information, event information (1220), and shooting date (1230) regarding the people (1210) included in the social event database (1200). In the embodiment illustrated in FIG. 12, the personalized social event database (1250) may include information regarding the face images of the people (1210-1, 1210-2, 1210-3) and the shooting date (1252) of the images.

[0147] FIG. 13 is a diagram illustrating the operation of an electronic device (100) according to one embodiment of the present disclosure acquiring notification content (1310) recommending a social event and displaying the notification content (1310).

[0148] Referring to FIG. 13, the electronic device (100) may acquire notification content (1310) recommending a social event and display the notification content (1310) on a display (140). In one embodiment of the present disclosure, the processor (120, see FIG. 3) of the electronic device (100) may generate a personalized social event by executing instructions or program code of a social event generation module (134). The personalized social event may include information regarding at least one of, for example, a person (e.g., 'Friend 1'), a social event (e.g., 'Meeting'), and whether it is a group (e.g., 'Individual'). Information regarding the personalized social event may be input to the notification content generation module (135). The processor (120) may generate notification content (1310) recommending a personalized social event by executing instructions or program code of the notification content generation module (135).

[0149] In one embodiment of the present disclosure, the notification content generation module (135) may include a notification message database (1300). The notification message database (1300) may store a plurality of notification message templates (1301, 1302, ...). The plurality of notification message templates (1301, 1302, ...) may consist of notification messages pre-written to recommend social events. The plurality of notification message templates (1301, 1302, ...) may be classified by the recommended social event. The plurality of notification message templates (1301, 1302, ...) may be classified by social event, for example, meeting notifications or birthday notifications. The first notification message template (1301) for meeting notifications may include, for example, a notification message recommending a meeting with an individual, such as "It’s been a while since we’ve been in touch with {}. How about reaching out for the first time in a while?" (1301-1) and a notification message recommending a meeting with a group the user belongs to, such as "It’s been a long time since we’ve been in touch with {}. How about making plans for the first time in a while?" (1301-2) and a notification message recommending a meeting with a group the user belongs to, such as "It’s been a while since we’ve met {}. How about making plans for the first time in a while?" (1301-3). The second notification message template (1302) for birthday notifications may include, for example, a notification message recommending a birthday party, such as "( )’s birthday is approaching. How about planning a birthday party?" (1302-1) and a notification message recommending a gift, such as "( )’s birthday is approaching. How about preparing a gift?" (1302-2).

[0150] The processor (120) can select a notification message corresponding to information on a personalized social event from among a plurality of notification message templates (1301, 1302, ...) already stored in the notification message database (1300). In the embodiment illustrated in FIG. 13, the processor (120) can select a corresponding notification message (1301-1) from among the notification messages (1301-1, 1301-2, 1301-3, ..., 1302-1, 1302-2, ...) included in the plurality of notification message templates (1301, 1302, ...) based on a friend 1, a meeting, and an individual included in the personalized social event information.

[0151] The processor (120) can obtain a notification message (1312) that recommends a personalized social event based on a selected notification message (1301-1). The notification message (1312) may be, for example, "It's Friend 1, whom you haven't been in touch with lately. How about reaching out for the first time in a while?" The processor (120) can display notification content (1310) containing the notification message (1312) and a representative image (1311) of the first person through a display (140).

[0152] FIG. 14 is a diagram illustrating the operation of an electronic device (100) according to one embodiment of the present disclosure generating notification content (1410) that recommends a social event and displaying the notification content (1410).

[0153] Referring to FIG. 14, the electronic device (100) can generate notification content (1410) recommending a social event and display the notification content (1410) on a display (140). In one embodiment of the present disclosure, the processor (120, see FIG. 3) of the electronic device (100) can generate a personalized social event by executing instructions or program code of a social event generation module (134). The personalized social event may include information regarding at least one of, for example, a person (e.g., 'Friend 1'), a social event (e.g., 'Meeting'), and whether it is a group (e.g., 'Individual'). Information about the personalized social event may be input into the notification content generation module (135). The processor (120) can generate notification content (1310) recommending a personalized social event by executing instructions or program code of the notification content generation module (135).

[0154] In one embodiment of the present disclosure, the notification content generation module (135) may include a generative artificial intelligence model (1400). The generative artificial intelligence model (1400) may be implemented as an artificial intelligence model trained to generate a response message for an input prompt. In one embodiment of the present disclosure, the generative artificial intelligence model (1400) may be a large language model (LLM) that is pre-trained based on vast text data. The large language model may be implemented as a transformer that is composed of an artificial neural network having numerous parameters (e.g., billions of weights or more) and is trained to output a response by discovering patterns, structures, characteristics, etc. of text using a large amount of unlabeled text through an unsupervised learning method. The large language model may provide a response by processing and analyzing the input text data.

[0155] In one embodiment of the present disclosure, a processor (120, see FIG. 3) of an electronic device (100) can generate a prompt using information regarding at least one of a person (e.g., 'Friend 1'), a social event (e.g., 'meeting'), and whether it is a group (e.g., 'individual') included in a personalized social event, and input the prompt into a generative artificial intelligence model (1400). The prompt may be composed of text data such as, for example, "Generate a message recommending a meeting with Friend 1 after a long time~". The processor (120) can generate a response message corresponding to the prompt by analyzing and processing the prompt using the generative artificial intelligence model (1400). The response message is a notification message (1412) recommending a personalized social event, for example, "It's Friend 1 I haven't been in touch with lately. How about reaching out after a long time?"

[0156] The processor (120) can display notification content (1410) including a generated notification message (1412) and a representative image (1411) of the first person through the display (140).

[0157] The present disclosure provides a method for an electronic device (100) to recommend a social event. The method of operation of the electronic device (100) may include a step (S210) of obtaining person information regarding each of the people from a plurality of previously stored images. The method of operation of the electronic device (100) may include a step (S220) of obtaining event information from a plurality of images. The method of operation of the electronic device (100) may include a step (S230) of obtaining at least one of a shooting date and time and a shooting location from the metadata of each of the plurality of images. The method of operation of the electronic device (100) may include a step (S240) of generating a social event related to at least one of the people based on at least one of the person information, event information, shooting date and time, and shooting location. The method of operation of the electronic device (100) may include a step (S250) of providing notification content including recommendation information regarding the social event.

[0158] In one embodiment of the present disclosure, the step of acquiring person information may include the step of grouping people recognized from a plurality of images to acquire a person database containing image information for each person.

[0159] In one embodiment of the present disclosure, the step of acquiring the person database may include: detecting bounding boxes representing the faces of people from a plurality of images; extracting face feature vectors from the images of the detected bounding boxes; grouping the people by performing face clustering based on the similarity of the extracted face feature vectors; and generating a person database including image information for each of the grouped people.

[0160] In one embodiment of the present disclosure, the step of acquiring the person database may include acquiring a first person database (S710) that includes information regarding at least one image grouped as a first person among the people recognized from a plurality of images. The step of acquiring the metadata (S230) may include acquiring information regarding the most recent shooting date from the metadata of at least one image included in the first person database (S720). The step of acquiring the social event (S240) may include determining whether there has been a meeting with the first person based on the number of at least one image included in the first person database and information regarding the most recent shooting date (S750, S760). The step of acquiring the social event (S240) may include generating a social event that recommends a meeting or contact with the first person based on the determination result (S770).

[0161] In one embodiment of the present disclosure, the step of acquiring the social event (S240) may further include: a step of identifying the number of images in which the first person and the second person are recognized together among at least one image (S910); a step of acquiring information regarding the most recent shooting date from a second person database regarding at least one image grouped as the second person (S930); and a step of determining whether there has been a meeting between the first person and the second person based on the number of identified images and the information regarding the most recent shooting date (S950, S960). The step of acquiring the social event (S240) may further include a step (S970) of generating a social event that recommends a meeting or contact with the first person and the second person based on the determination result.

[0162] In one embodiment of the present disclosure, the method of operation of the electronic device (100) may further include the step (S1110) of identifying an image related to a social event among a plurality of images based on a tag including the time and place of shooting of each of the plurality of images. The step of acquiring the social event (S240) may include the step (S1120) of recognizing a person related to the social event from the identified image using an object recognition model; and the step (S1140) of generating a social event related to the recognized person.

[0163] In one embodiment of the present disclosure, the step of acquiring the social event (S240) may further include: a step of grouping at least one person recognized from the identified image by person through face clustering; a step of acquiring information regarding the shooting date from the metadata of the identified image; and a step of creating a social event database using the information regarding the grouped at least one person, tags, and shooting date.

[0164] In one embodiment of the present disclosure, the step of providing the notification content (S250) may include: selecting a notification message template corresponding to a social event from a plurality of notification message templates stored in a notification message database; and obtaining a notification message recommending a social event using the selected notification message template. The step of displaying the notification content (S250) may include displaying the obtained notification message together with a representative image.

[0165] In one embodiment of the present disclosure, the step of providing the notification content (S250) may include the step of generating a notification message recommending a social event using a generative artificial intelligence model trained to generate a response message to an input prompt. The step of displaying the notification content (S250) may include the step of displaying the generated notification message together with a representative image.

[0166] The present disclosure provides an electronic device (100) for recommending social events. The electronic device (100) may include a storage (110) for storing a plurality of images; at least one processor (120) including a processing circuit; a memory (130) for storing one or more instructions; and a display (140). By executing the one or more instructions individually or collectively by the at least one processor (120), the electronic device (100) can obtain person information regarding each of the people from the plurality of images already stored in the storage (110). By executing the one or more instructions individually or collectively by the at least one processor (120), the electronic device (100) can obtain event information from the plurality of images. By executing the above one or more commands individually or collectively by at least one processor (120), the electronic device (100) can obtain at least one of the shooting date and time and the shooting location from the metadata of each of the plurality of images. By executing the above one or more commands individually or collectively by at least one processor (120), the electronic device (100) can obtain a social event related to at least one of the people based on at least one of person information, event information, the shooting date and time and the shooting location. By executing the above one or more commands individually or collectively by at least one processor (120), the electronic device (100) can provide notification content including recommendation information regarding the social event through the display (140).

[0167] In one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can group people recognized from a plurality of images and obtain a person database including person-specific image information.

[0168] In one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can detect bounding boxes representing the faces of people from a plurality of images, extract face feature vectors from the images of the detected bounding boxes, and perform face clustering based on the similarity of the extracted face feature vectors, thereby grouping the people and creating a person database containing image information for each of the grouped people.

[0169] In one embodiment of the present disclosure, the person database may include person identification information, the file name of at least one image grouped as a person, the storage location of at least one file, the location coordinates of a bounding box, and information regarding at least one of an event recognized from at least one image.

[0170] In one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) may acquire a first person database containing information regarding at least one image grouped as a first person among people recognized from a plurality of images, acquire information regarding the most recent shooting date from the metadata of at least one image included in the first person database, and determine whether there has been a meeting with the first person based on the number of at least one image included in the first person database and the information regarding the most recent shooting date. By executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) may generate a social event recommending a meeting or contact with the first person based on the result of the determination.

[0171] In one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can identify the number of images in which a first person and a second person are recognized together among at least one image, obtain information regarding the most recent shooting date from a second person database regarding at least one image grouped with the second person, and determine whether there has been a meeting between the first person and the second person based on the number of identified images and the information regarding the most recent shooting date. By executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can generate a social event recommending a meeting or contact with the first person and the second person based on the result of the determination.

[0172] In one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can identify an image related to a social event among a plurality of images based on a tag including the time and place of each of the plurality of images, recognize a person related to the social event from the identified image using an object recognition model, and generate a social event related to the recognized person.

[0173] In one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can group at least one person recognized from an identified image by person through face clustering, obtain information regarding the shooting date from the metadata of the identified image, and create a social event database using the information regarding at least one grouped person, tag, and shooting date.

[0174] In one embodiment of the present disclosure, the notification content may include a representative image related to a social event and a notification message recommending the social event.

[0175] In one embodiment of the present disclosure, by executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can select a notification message template corresponding to a social event from a plurality of notification message templates stored in a notification message database, and obtain a notification message recommending a social event using the selected notification message template. By executing one or more instructions individually or collectively by at least one processor (120), the electronic device (100) can display the notification message along with a representative image through a display (140).

[0176] In one embodiment of the present disclosure, by executing one or more commands individually or collectively by at least one processor (120), the electronic device (100) can generate a notification message recommending a social event using a generative artificial intelligence model trained to generate a response message to an input prompt. By executing one or more commands individually or collectively by at least one processor (120), the electronic device (100) can display the notification message along with a representative image through a display (140).

[0177] The present disclosure provides a computer program product comprising a computer-readable storage medium. The storage medium may include instructions readable by the electronic device (100) to enable the electronic device (100) to perform the following operations: obtaining person information regarding each of the people from a plurality of images stored in the electronic device (100); obtaining event information from the plurality of images; obtaining at least one of a shooting date and time and a shooting location from the metadata of each of the plurality of images; obtaining a social event related to at least one of the people based on at least one of the person information, event information, shooting date and time and a shooting location; and providing notification content including recommendation information regarding the social event.

[0178] A program executed by the electronic device (100) described in the present disclosure may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. The program may be executed by any system capable of executing computer-readable instructions.

[0179] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively.

[0180] Software can be implemented as a computer program containing instructions stored on a computer-readable storage medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROMs, DVDs (Digital Versatile Discs)). Computer-readable recording media can be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. The medium is readable by a computer, stored in memory, and can be executed by a processor.

[0181] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory' means only that the storage medium does not contain a signal and is tangible, and does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0182] In addition, the program according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.

[0183] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, the computer program product may be from the manufacturer of the electronic device (100) or an electronic market (e.g., Samsung Galaxy Store). TMIt may include a product in the form of a software program that is distributed electronically through ). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a server of the manufacturer of the electronic device (100), a server of an electronic market, or a storage medium of a relay server that temporarily stores the software program.

[0184] A computer program product may include a storage medium of a server or a storage medium of an electronic device (100) in a system composed of an electronic device (100) and / or a server. Alternatively, if there is a third device that is communicationally connected to the electronic device (100), the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself that is transmitted from the electronic device (100) to the third device or from the third device to the electronic device.

[0185] In this case, either the electronic device (100) or one of the third devices may execute a computer program product to perform the method according to the disclosed embodiments. Alternatively, at least one of the electronic device (100) and the third device may execute a computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0186] For example, an electronic device (100) can execute a computer program product stored in memory (130, see FIG. 3) to control another electronic device that is connected to the electronic device (100) to perform a method according to the disclosed embodiments.

[0187] As another example, a third device may execute a computer program product to control an electronic device connected to the third device in communication to perform the method according to the disclosed embodiment.

[0188] When the third device executes a computer program product, the third device may download the computer program product from the electronic device (100) and execute the downloaded computer program product. Alternatively, the third device may execute a computer program product provided in a pre-loaded state to perform the method according to the disclosed embodiments.

[0189] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, appropriate results can be achieved even if the described techniques are performed in a different order than described, and / or components such as the described computer system or module are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

Claims

1. In a method for an electronic device (100) to recommend a social event, A step of obtaining person information regarding each person from a plurality of images stored in the electronic device (100); A step of obtaining event information from the above plurality of images (S220); A step (S230) of obtaining at least one of the shooting date and time and the shooting location from the metadata of each of the plurality of images; A step (S240) of obtaining a social event related to at least one of the people based on at least one of the person information, the event information, the shooting time and location; and A step (S250) of providing notification content including recommendation information regarding the above social event; A method including 2. In Paragraph 1, The step of obtaining the above-mentioned person information is, A method comprising the step of grouping the individuals recognized from the plurality of images to obtain a person database containing image information for each individual.

3. In Paragraph 2, The step of acquiring the above-mentioned person database is, The method includes the step (S710) of obtaining a first person database comprising information regarding at least one image grouped as a first person among the people recognized from the plurality of images; and The step of acquiring the above metadata (S230) is, The step (S720) of obtaining information regarding the most recent shooting date from the metadata of at least one image included in the first person database; The step (S240) of acquiring the above social event is, A step of determining whether there was a meeting with the first person based on the number of at least one image included in the first person database and information regarding the most recent shooting date (S750, S760); and A step (S770) of creating a social event that recommends meeting or contacting the first person based on the result of the judgment; A method including 4. In any one of paragraphs 1 to 3, A step (S1110) of identifying an image related to a social event among the plurality of images based on a tag including the shooting date and location of each of the plurality of images; Includes more, The step (S240) of acquiring the above social event is, A step (S1120) of recognizing a person related to the social event from the identified image using an object recognition model; and Step of generating a social event related to the above-mentioned recognized person (S1140); A method including 5. In any one of paragraphs 1 through 4, A method in which the above notification content includes a representative image related to the above social event and a notification message recommending the above social event.

6. In Paragraph 5, The step (S250) of providing the above notification content is, A step of selecting a notification message template corresponding to the social event from a plurality of notification message templates stored in a notification message database; A step of obtaining a notification message recommending the social event using the selected notification message template; and A step of displaying the above-mentioned acquired notification message together with the above-mentioned representative image; A method including 7. In Paragraph 5, The step (S250) of providing the above notification content is, A step of generating the notification message recommending the social event using a generative artificial intelligence model trained to generate a response message to an input prompt; and A step of displaying the generated notification message together with the representative image; A method including 8. In an electronic device (100) that recommends social events, Storage (110) for storing multiple images; At least one processor (120) including a processing circuit; Memory (130) for storing one or more instructions; and Display (140); Includes, By executing the above one or more instructions individually or collectively by the at least one processor (120), the electronic device (100) Person information regarding each person is obtained from the plurality of images already stored in the storage (110), and Obtaining event information from the above plurality of images, and At least one of the shooting date and time and the shooting location is obtained from the metadata of each of the plurality of images above, and Based on at least one of the above person information, above event information, above shooting time and date, and above shooting location, a social event related to at least one of the above people is obtained, and An electronic device (100) that provides notification content including recommendation information regarding the social event through the display (140).

9. In Paragraph 8, By executing the above one or more instructions individually or collectively by the above at least one processor (120), the electronic device (100) An electronic device (100) that obtains a person database containing image information for each person by grouping the persons recognized from the plurality of images.

10. In Paragraph 9, By executing the above one or more instructions individually or collectively by the above at least one processor (120), the electronic device (100) A first person database is obtained that includes information regarding at least one image grouped as a first person among the persons recognized from the plurality of images above, and Information regarding the most recent shooting date is obtained from the metadata of at least one image included in the first person database, and Determining whether there was a meeting with the first person based on the number of at least one image included in the first person database and information regarding the most recent shooting date, An electronic device (100) that generates a social event recommending a meeting or contact with the first person based on the result of a judgment.

11. In any one of paragraphs 8 through 10, By executing the above one or more instructions individually or collectively by the above at least one processor (120), the electronic device (100) Based on tags including the shooting date and location of each of the plurality of images, an image related to a social event among the plurality of images is identified, and Using an object recognition model, recognize a person related to the social event from the identified image, and An electronic device (100) that generates social events related to the above-mentioned recognized person.

12. In any one of paragraphs 8 through 11, The above notification content includes a representative image related to a social event and a notification message recommending the social event, an electronic device (100).

13. In Paragraph 12, By executing the above one or more instructions individually or collectively by the above at least one processor (120), the electronic device (100) Select the notification message template corresponding to the social event from a plurality of notification message templates already stored in the notification message database, and Obtain a notification message recommending the social event using the selected notification message template above, and An electronic device (100) that displays the acquired notification message along with the representative image through the display (140).

14. In Paragraph 12, By executing the above one or more instructions individually or collectively by the above at least one processor (120), the electronic device (100) Using a generative artificial intelligence model trained to generate a response message to an input prompt, the above notification message recommending the above social event is generated, and An electronic device (100) that displays the generated notification message along with the representative image through the display (140).

15. A computer-readable recording medium having at least one program recorded thereon for implementing the method described in any one of claims 1 to 7.

Images