Electronic device, operation method thereof, and storage medium

Abstract

An electronic device is disclosed. The electronic device comprises: at least one processor including a processing circuit; and a memory storing instructions and including one or more storage media, wherein the instructions, when executed individually or collectively by the at least one processor, control the electronic device to: when an image capture trigger operation is identified, acquire a captured image; when an object in the acquired captured image is identified, input the acquired image into a trained first neural network model to generate an image corresponding to the identified object; and register the generated image as content corresponding to a user.

Description

Electronic devices and their operating methods and storage media

[0001] The present disclosure relates to an electronic device and an operation method and a storage medium thereof, and more particularly, to an electronic device and an operation method and a storage medium thereof that registers content using an image taken of an object such as clothing or food, or provides sound related to the content.

[0002] Advances in electronic technology have led to the development and widespread adoption of various types of electronic devices. Recently, clothing managers, which register and store information about the user's clothing for convenient clothing management, have become widely used. Furthermore, refrigerators are being developed and distributed, offering convenient food management features that allow users to register information about the food they store or plan to store.

[0003] An electronic device according to one embodiment of the present disclosure includes at least one processor including a processing circuit and a memory storing instructions and including one or more storage media, wherein the instructions, when individually or collectively executed by the at least one processor, can control the electronic device to acquire a captured image when a capture trigger operation is identified.

[0004] According to one embodiment, the instructions may control, when an object is identified in the acquired captured image, to input the acquired image into a trained first neural network model to generate an image corresponding to the identified object.

[0005] According to one embodiment, the instructions may control registering the generated image as content corresponding to a user.

[0006] A method of operating an electronic device according to an embodiment of the present disclosure may include an operation of acquiring a shooting image when a shooting trigger operation is identified.

[0007] An operating method of an electronic device according to an embodiment of the present disclosure may include an operation of, when an object is identified in the acquired photographed image, inputting the acquired image into a learned first neural network model to generate an image corresponding to the identified object.

[0008] A method of operating an electronic device according to an embodiment of the present disclosure may include an operation of registering the generated image as content corresponding to a user.

[0009] In a storage medium storing computer-readable instructions according to one embodiment of the present disclosure, the instructions, when executed by at least one processor of an electronic device, can control the electronic device to acquire a photographed image when a photographing trigger operation is identified.

[0010] According to one embodiment, the instructions may control, when an object is identified in the acquired captured image, to input the acquired image into a trained first neural network model to generate an image corresponding to the identified object.

[0011] According to one embodiment, the instructions may control registering the generated image as content corresponding to a user.

[0012] FIG. 1A is a drawing schematically illustrating an example of use of an electronic device according to one embodiment.

[0013] FIG. 1b is a drawing schematically illustrating an example of use of an electronic device according to one embodiment.

[0014] Figure 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0015] Figure 3 is a flowchart for explaining a control method of an electronic device according to one embodiment.

[0016] FIG. 4a is a flowchart illustrating a method for registering a clothing image according to one embodiment.

[0017] FIG. 4b is a drawing for explaining a method for registering a clothing image according to one embodiment.

[0018] Figure 5a is a flowchart for explaining an occlusion area according to one embodiment.

[0019] FIG. 5b is a drawing for explaining an occlusion area according to one embodiment.

[0020] FIG. 6 is a flowchart illustrating a method for generating a clothing image according to one embodiment.

[0021] FIG. 7 is a flowchart illustrating a method for registering clothing images based on a user's selection according to one embodiment.

[0022] FIG. 8 is a flowchart illustrating a method for obtaining sound associated with an object according to one embodiment.

[0023] FIG. 9 is a flowchart illustrating a method of outputting acquired sound through a speaker according to one embodiment.

[0024] Fig. 10 is a flowchart illustrating a method of outputting acquired sound through a speaker according to one embodiment.

[0025] FIG. 11 is a diagram for explaining a first neural network model according to one embodiment.

[0026] FIG. 12 is a diagram for explaining a first neural network model according to one embodiment.

[0027] FIG. 13 is a diagram for explaining a second neural network model according to one embodiment.

[0028] Fig. 14 is a block diagram illustrating a learning method of a second neural network model according to one embodiment.

[0029] Fig. 15 is a block diagram showing a detailed configuration of an electronic device according to one embodiment.

[0030] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0031] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0032] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0033] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0034] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0035] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0036] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0037] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0038] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0039] And in this specification, the term "signal" includes not only an electrical signal but also a signal in the form of a sound wave, and in the case of an electrical signal, it may be a digital signal as well as an analog signal. For example, the expression "audio signal" (or noise signal) means a sound wave (or radio wave) signal if the signal is external to the electronic device, and an electrical signal if the signal is internal to the electronic device, depending on the location. In addition, signal processing within the electronic device described below may be not only a digital signal processing, but also an analog signal processing, or a signal processing method that uses a mixture of analog and digital methods.

[0040] And in this specification, the term "filter" means removing a specific component (e.g., a specific frequency range or a specific pattern), and the filter may be a digital filter or an analog filter.

[0041] FIG. 1A and FIG. 1B are drawings schematically illustrating an example of use of an electronic device according to one embodiment.

[0042] According to FIGS. 1A and 1B, according to one embodiment, an electronic device (10-1 or 10-2) can generate an image and register it as content.

[0043] For example, the electronic device (10-1 or 10-2) may be implemented as a clothing manager (10-1). For example, when a photographed image corresponding to clothing is acquired, the electronic device (10-1) may generate a clothing image based on the acquired image. For example, when an image of a user (1) holding a clothing or an image of a user (1) wearing a clothing is acquired, the electronic device (10-1) may generate a clothing image using the acquired image.

[0044] For example, an electronic device may register a generated clothing image as content corresponding to a user. For example, the content corresponding to a user may be clothing stored in a clothing manager and its corresponding image, but is not limited thereto. It may also be clothing items owned by the user, including clothing scheduled to be stored in the clothing manager.

[0045] For example, the electronic device (10-1 or 10-2) may be implemented as a refrigerator (10-2). For example, when a photographed image corresponding to food is acquired, the electronic device (10-2) may generate a food image based on the acquired image. For example, when an image of food is acquired, the electronic device (10-2) may generate a food image using the acquired image.

[0046] For example, an electronic device may register a generated food image as content corresponding to a user. For example, the content corresponding to a user may be food stored or scheduled to be stored in a refrigerator and its corresponding image, but is not limited thereto. It may also be an item of food currently in the user's possession.

[0047] Alternatively, although not illustrated in FIGS. 1A and 1B , the electronic device (10-1 or 10-2) may be implemented as a server, according to an example. In an example, when a photographed image related to a different type of object, including clothing or food, is acquired from an external device (e.g., a user terminal), the electronic device may generate an image of the object based on the acquired image. In an example, the electronic device may transmit the generated image to a clothing manager or a refrigerator. The clothing manager or refrigerator may generate content corresponding to the user based on the received image. Alternatively, in an example, the electronic device may generate content corresponding to the user based on the generated image and transmit information about the content to the clothing manager or the refrigerator. However, for the convenience of explanation, the following description will be limited to a case where the electronic device is implemented as a clothing manager or a refrigerator.

[0048] For example, the electronic device (10-1 or 10-2) may acquire and output sounds related to clothing or food. For example, it may be assumed that the electronic device is implemented as a clothing manager (10-1). The electronic device (10-1) may acquire and output sounds when a user takes out or puts in clothes from the electronic device (10-1). For example, the output sounds may be sounds generated based on the current weather, the user's schedule, or the type of clothing.

[0049] Figure 2 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0050] According to FIG. 2, the electronic device (100) may include at least one processor (110) and memory (120).

[0051] According to an example, the electronic device (100) may be implemented as a clothing manager (or an air dresser). Alternatively, the electronic device (100) may be implemented as a refrigerator. Alternatively, according to an example, the electronic device (100) may be implemented as a server. However, the present invention is not limited thereto, and the electronic device (100) may be implemented as a different type of electronic device than the clothing manager or the refrigerator.

[0052] At least one processor (110) (hereinafter, “processor”) is electrically connected to the memory (120) and controls the overall operation of the electronic device (100). The processor (110) may be composed of one or more processors. Specifically, the processor (110) may perform operations of the electronic device (100) according to various embodiments of the present disclosure by executing at least one instruction stored in the memory (120).

[0053] According to one embodiment, the processor (110) may be implemented as a digital signal processor (DSP), a microprocessor, a graphics processing unit (GPU), an artificial intelligence (AI) processor, a neural processing unit (NPU), or a time controller (TCON) for processing a digital image signal. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), or an ARM processor, or may be defined by the relevant terms. In addition, the processor (110) may be implemented as a system on chip (SoC) or large scale integration (LSI) having a built-in processing algorithm, or may be implemented in the form of an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA).

[0054] The memory (120) can store data required for various embodiments. The memory (120) may be implemented in the form of a memory embedded in the electronic device (100) or may be implemented in the form of a memory that can be detachably attached to the electronic device (100) depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding the functions of the electronic device (100) may be stored in a memory that can be detachably attached to the electronic device (100).

[0055] Meanwhile, in the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)). In addition, in the case of memory that can be detachably attached to the electronic device (100), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented.

[0056] According to one embodiment, the processor (110) may identify a shooting trigger operation. According to one example, the shooting trigger operation may be an operation that identifies the presence of a user at a preset location (e.g., front) from the electronic device (100). According to one example, the processor (110) may receive information (e.g., an image) about the preset location from the electronic device (100) from an external device (e.g., a user terminal or a camera) through a communication interface (e.g., the communication interface (160) of FIG. 15). According to one example, the processor (110) may monitor the presence of the user in real time based on the information received from the external device.

[0057] For example, the shooting trigger action may be an action that identifies an object as existing at a preset location from the electronic device (100). For example, if the electronic device (100) is implemented as a clothing manager, the processor (110) may identify the presence of an object by monitoring whether clothing is present at a preset location from the clothing manager. In another example, the clothing may include accessories. Alternatively, for example, if the electronic device (100) is implemented as a refrigerator, the processor (110) may identify whether food is present at a preset location from the refrigerator.

[0058] According to one embodiment, the processor (110) may acquire a captured image. According to one example, the electronic device (100) may be in a state of communication connection with an external device. According to one example, the external device may be a device capable of capturing an image, such as a user terminal or a camera. According to one example, the processor (110) may acquire the captured image from the external device via a communication interface. According to one example, the captured image may be an image including an object. For example, the captured image may be an image including a user. Alternatively, the captured image may be an image including objects of different times, including clothing or food. According to one example, when a capture trigger operation is identified, the processor (110) may acquire the captured image from the external device.

[0059] According to one embodiment, the processor (110) can identify an object within the acquired captured image. For example, the processor (110) can identify the presence of an object within the captured image using a designated algorithm. For example, the processor (110) can confirm the presence of a person within the captured image. Alternatively, if the electronic device (100) is implemented as a clothing manager, the processor (110) can confirm the presence of clothing within the captured image. Alternatively, if the electronic device (100) is implemented as a refrigerator, the processor (110) can confirm the presence of food within the captured image.

[0060] In one embodiment, the processor (110) may input a captured image into a trained first neural network model to generate an image corresponding to the identified object. In one example, the first neural network model may be a model trained to output an image of an object included in the captured image when the captured image is input. For example, even if clothing is present in the captured image, but a portion of the clothing is cut off or obscured, the first neural network model may output various images of the clothing in the captured image when the captured image is input.

[0061] For example, if the captured image includes clothing, the first neural network model may output an image for registering clothing as content. Alternatively, if the captured image includes food, the first neural network model may output an image for registering food as content. Accordingly, an image different from the captured image may be output. The first neural network model will be described in detail with reference to FIGS. 11 and 12.

[0062] For example, the processor (110) may acquire sounds related to objects included in a captured image using a learned neural network model. For example, the processor (110) may output the acquired sounds when a user performs an action related to the object. This will be described in detail with reference to FIGS. 8, 9, 10, 13, and 14.

[0063] According to one embodiment, the processor (110) may register the generated image as content corresponding to the user. According to one example, when the electronic device (100) is implemented as a clothing manager, the content corresponding to the user may be, but is not limited to, at least one piece of clothing owned by the user and an image corresponding thereto. According to one example, the content corresponding to the user may be, but is not limited to, clothing currently stored in the clothing manager and an image corresponding thereto. Alternatively, according to one example, when the electronic device (100) is implemented as a refrigerator, the content corresponding to the user may be, but is not limited to, at least one piece of food stored or scheduled to be stored in the refrigerator and an image corresponding thereto. According to one example, the content corresponding to the user may be stored in the memory (120), but is not limited thereto. The content corresponding to the user may be stored in an external device (e.g., a server).

[0064] Figure 3 is a flowchart for explaining an operating method of an electronic device according to one embodiment.

[0065] According to FIG. 3, according to one embodiment, the operating method may include an operation (S310) of acquiring a shooting image when a shooting trigger operation is identified.

[0066] In one example, an electronic device (e.g., the electronic device (100) of FIG. 2) can identify a shooting trigger operation (e.g., the shooting trigger operation of FIG. 2). In one example, the electronic device can acquire a shooting image when it is determined that a user is present at a preset location from the electronic device. In one example, the electronic device can acquire a shooting image from an external device through a communication interface (e.g., the communication interface (160) of FIG. 15).

[0067] According to one embodiment, the operating method may include an operation (S320) of, when an object is identified in an acquired captured image, inputting the captured image into a learned first neural network model to generate an image corresponding to the identified object.

[0068] In one example, an electronic device can identify whether an object (e.g., the object of FIG. 2) exists in a captured image. In one example, the object may be, but is not limited to, clothing or food. In one example, the electronic device can identify the object in the image using a specified algorithm. In one example, when the object in the captured image is identified, the electronic device can input the captured image into a trained first neural network model (e.g., the first neural network model of FIG. 2) to generate an image of the object. In one example, the image of the object may be a regenerated image of the object included in the image.

[0069] According to one embodiment, the operating method may include an operation (S330) of registering a generated image as content corresponding to a user. According to one example, when an image for an object is generated, the electronic device may register it as content corresponding to a user (e.g., content corresponding to a user of FIG. 2). For example, if the object is clothing, the electronic device may add the identified clothing image to a list of content corresponding to the user, and store information related to the clothing image (e.g., the type of clothing) in a memory (e.g., memory (120) of FIG. 2).

[0070] FIGS. 4A and 4B are a flowchart and diagram illustrating a method for registering a clothing image according to one embodiment.

[0071] According to FIGS. 4A and 4B, according to one embodiment, the operating method may include an operation (S410) of acquiring a photographed image (410, e.g., the photographed image of FIG. 2) related to clothing when a photographing trigger operation is identified.

[0072] For example, an electronic device (e.g., the electronic device (100) of FIG. 2) may first identify a shooting trigger operation (e.g., the shooting trigger operation of FIG. 2). For example, when the shooting trigger operation is identified, the electronic device may acquire a shooting image (410) related to clothing. For example, it may be assumed that the electronic device is implemented as a clothing manager. When the electronic device identifies that clothing (or a user wearing or scheduled to wear clothing) is present at a preset location (e.g., in front of the clothing manager), the electronic device may acquire a shooting image (410) related to clothing at the preset location.

[0073] According to one embodiment, the operating method may include an operation (S420) of obtaining skeleton information of the identified object when an object (e.g., an object of FIG. 2) in a photographed image (410) related to clothing is identified.

[0074] For example, when a captured image (410) is acquired, the electronic device can identify whether an object exists within the captured image (410). For example, the electronic device can determine whether clothing exists within the captured image (410). As illustrated in FIG. 4A, when multiple pieces of clothing exist within the captured image (410), the electronic device can identify each of the multiple pieces of clothing. For example, when multiple pieces of clothing (e.g., tops and bottoms) exist within the captured image (410), the electronic device can segment and identify each piece of clothing.

[0075] In one example, if the electronic device identifies that clothing exists in the captured image (410), the electronic device may obtain skeleton information of the identified object. In one example, the electronic device may obtain skeleton information (421 and 422) corresponding to the clothing using a specified algorithm. In one example, the electronic device may identify first skeleton information (421) corresponding to the first clothing based on an image of an area (411) corresponding to the first clothing. In one example, the electronic device may identify second skeleton information (422) corresponding to the second clothing based on an image of an area (412) corresponding to the second clothing. In one example, the area (411) corresponding to the first clothing and the area (412) corresponding to the second clothing may include a common area.

[0076] For example, the area corresponding to each garment may not include the overall shape of the garment. Accordingly, an occlusion area may exist for each garment. For example, the occlusion area may be an area within the area corresponding to the garment that does not include the shape of the garment. For example, if a portion of a portion corresponding to the left arm of the first garment is not included in the area (411) corresponding to the first garment, an occlusion area (431) may exist in the portion corresponding to the left arm of the first garment.

[0077] In one example, the electronic device can identify occluded areas (431, 432, and 433) from skeleton information using a specified algorithm. In one example, the electronic device can identify occluded areas (431, 432, and 433) corresponding to each of a plurality of garments. This will be described in detail with reference to FIGS. 5A and 5B .

[0078] According to one embodiment, the operating method may include an operation (S430) of inputting at least one of the acquired skeleton information and the acquired photographed image (410) into a learned first neural network model (e.g., the first neural network model of FIG. 2) to generate a clothing image corresponding to the identified object.

[0079] In one example, when skeleton information is acquired, the electronic device may input the acquired skeleton information and the acquired photographed image (410) into a trained first neural network model to generate an image of clothing included in the photographed image (410). In one example, the clothing image output through the first neural network model may be an image including the overall shape of the clothing. In one example, the generated clothing image may be a 2D (Dimensional) image, but is not limited thereto, and may also be a 3D image. In one example, the electronic device may input information on an occluded area together with the photographed image (410) and skeleton information into the first neural network model to generate an image of clothing.

[0080] For example, the electronic device can input first skeleton information (421) and a photographed image (410) corresponding to the first garment into a trained first neural network model to obtain a garment image for the first garment. Alternatively, for example, the electronic device can input second skeleton information (422) and a photographed image (410) corresponding to the second garment into a trained second neural network model to obtain a garment image for the second garment. In one example, the electronic device can input first skeleton information (421) and second skeleton information (422) corresponding to the first garment into the first neural network model, respectively, to obtain a garment image for the first garment and a garment image for the second garment, respectively. A specific method for obtaining garment images will be described in detail with reference to FIGS. 11 and 12.

[0081] According to one embodiment, the operating method may include an operation (S440) of registering the generated clothing image as a clothing image corresponding to the user.

[0082] For example, the electronic device may register the generated clothing image as a clothing image corresponding to the user, along with information about the clothing. For example, the information about the clothing may include information about the category of the clothing. For example, if clothing images corresponding to the first clothing and the second clothing included in the captured image (410) are generated, the electronic device may register the clothing image corresponding to the first clothing and the clothing image corresponding to the second clothing as clothing images corresponding to the user. For example, the electronic device may register the first clothing and the second clothing as clothing owned by the user.

[0083] According to the above example, by inputting the skeletal information of the clothing included in the captured image into a trained neural network model, the electronic device can obtain an image accurately representing the texture and shape of the clothing the user wishes to register. Furthermore, even if a portion of the clothing included in the captured image is obscured or cropped, the device can obtain an image of the clothing that includes its overall shape, thereby increasing user convenience.

[0084] FIG. 5a and FIG. 5b are a flowchart and diagram for explaining an occlusion area according to one embodiment.

[0085] According to FIGS. 5A and 5B , according to an embodiment, the operating method may include an operation (S510) of identifying an occluded area (531, e.g., the occluded area of ​​FIGS. 4A and 4B ) corresponding to an object based on the shape of the object (e.g., the object of FIG. 4A ). According to an example, the shape of the object may be, but is not limited to, the overall shape of the object, and may include at least one of the shape, appearance, and color of the object.

[0086] For example, when an electronic device (e.g., the electronic device (100) of FIG. 2) acquires a captured image (520), it may acquire skeleton information (530) based on an area corresponding to an object. For example, if the area corresponding to the object does not include the overall shape of the object, the electronic device may identify the area where the object is not included as an occluded area (531). Alternatively, for example, if a portion of the object is occluded within the area corresponding to the object, the electronic device may identify the area where the object is not included as an occluded area (531).

[0087] In one example, the electronic device may identify an occlusion area (531) based on the acquired skeleton information (530) and sample skeleton information (540). For example, in the case of a top, the shape of the sample skeleton may be a 'T' shape. In one example, the electronic device may compare the sample skeleton information (540) with the skeleton information (530) of the identified clothing, and identify a portion of the skeleton of the identified clothing that is different from the shape of the sample skeleton (or an end portion of a different portion) as the occlusion area (531). In one example, it goes without saying that there may be a plurality of occlusion areas (531) corresponding to the identified clothing.

[0088] Alternatively, as an example, the electronic device may identify an occluded area (531) based on a specific body part of the user included in the captured image (520). For example, assume that the identified object is a lower garment. If the foot is not included in the area corresponding to the clothing, the electronic device may identify the location corresponding to the end of the lower garment as the occluded area (531).

[0089] According to one embodiment, the operating method may include an operation (S520) of inputting information about an identified occluded area (531), skeleton information (530), and a captured image (520) into a learned first neural network model (e.g., the first neural network model of FIG. 2) to generate a clothing image (e.g., the clothing image of FIG. 4A). According to one example, the information about the occluded area (531) may be position information (or coordinate information) corresponding to the occluded area (531) within the skeleton.

[0090] For example, when information about an identified occluded area (531), skeleton information (530), and a captured image (520) are input, the learned first neural network model can output an image including the entire shape of an object included in the captured image (520). By learning information about the occluded area (531) of the object together, the learned first neural network model can output an image including the entire shape of the object.

[0091] In one embodiment, the electronic device may provide a User Interface (UI) that requests recapture of an image related to clothing when a closed area corresponding to an object is identified. In one example, the electronic device may output information requesting recapture through a speaker (e.g., speaker (170) of FIG. 15 ) so that an image encompassing the overall shape of the clothing can be acquired. Alternatively, the electronic device may output information requesting recapture through a display (e.g., display (130) of FIG. 15 ).

[0092] As an example, the electronic device may generate an image of clothing using a recaptured image. For example, the electronic device may input the recaptured image along with the existing captured image (520) into a trained first neural network model to generate an image of clothing.

[0093] According to the above example, by inputting information about the occluded area associated with the garment the user wishes to register along with the captured image, the electronic device can obtain an image of the garment containing the overall shape of the garment, even if a portion of the garment included in the captured image is obscured or cropped. This can enhance user convenience and satisfaction.

[0094] FIG. 6 is a flowchart illustrating a method for generating a clothing image according to one embodiment.

[0095] According to FIG. 6, according to one embodiment, the operating method may include an operation (S610) of obtaining first skeleton information (e.g., first skeleton information (421) of FIG. 4b) of the identified first object when a first object (e.g., an object of FIG. 2) is identified in a captured image (e.g., a captured image (410) of FIG. 4b).

[0096] For example, it may be assumed that an electronic device (e.g., the electronic device (100) of FIG. 2) is implemented as a clothing manager. For example, the electronic device may identify multiple garments from a captured image, as illustrated in FIG. 4B. For example, when a first garment (e.g., a top) among the multiple garments is identified, the electronic device may obtain first skeleton information of the first garment from an image of an area corresponding to the identified first garment.

[0097] According to one embodiment, the operating method may include an operation (S620) of obtaining second skeleton information (e.g., second skeleton information (422) of FIG. 4b) of the identified second object (e.g., the object of FIG. 2) within the captured image.

[0098] In one example, the electronic device can identify multiple garments from a captured image, as illustrated in FIG. 4B . In another example, if a second garment is identified among the multiple garments, the electronic device can obtain second skeleton information of the second garment from an image of an area corresponding to the identified second garment.

[0099] According to one embodiment, the operating method may include an operation (S630) of inputting the acquired first skeleton information, the acquired second skeleton information, and the acquired photographed image into a learned first neural network model (e.g., the first neural network model of FIG. 2) to generate a first clothing image corresponding to a first object (e.g., and a second clothing image corresponding to a second object), respectively.

[0100] In one example, the electronic device may input first skeleton information and second skeleton information together with a captured image into a trained first neural network model. In one example, when the captured image, first skeleton information, and second skeleton information are input, the trained first neural network model may output at least one first clothing image corresponding to the first garment and at least one second clothing image corresponding to the second garment.

[0101] FIG. 7 is a flowchart illustrating a method for registering clothing images based on a user's selection according to one embodiment.

[0102] According to FIG. 7, according to one embodiment, the operating method may include an operation (S710) of inputting acquired skeleton information (e.g., skeleton information (421 and 422) of FIG. 4B) and acquired photographed image (e.g., photographed image (410) of FIG. 4B) into a learned first neural network model (e.g., the learned first neural network model of FIG. 2) to generate at least one clothing image corresponding to an identified object (e.g., an object of FIG. 2).

[0103] In one example, the trained first neural network model may be a model trained to output at least one clothing image corresponding to the identified clothing. For example, suppose a one-piece is identified in a photographed image. In one example, the trained first neural network model may generate different types of images of the one-piece included in the photographed image. For example, images of the one-piece from different viewpoints may be generated. However, this is not limited to this, and images of different styles of the one-piece may also be generated.

[0104] According to one embodiment, the operating method may include an operation (S720) of registering one of the selected clothing images as a clothing image corresponding to the user based on a user input for selecting one of the generated at least one clothing image.

[0105] In one example, the electronic device may provide a guide UI (User Interface) for selecting one of at least one generated clothing image. In one example, if the electronic device is implemented as a clothing manager or a refrigerator, the electronic device may provide the UI through a display included in the electronic device (e.g., the display (130) of FIG. 15). Alternatively, in one example, if the electronic device is implemented as a server, the electronic device may transmit information about the UI to an external device (e.g., the clothing manager or the refrigerator) through a communication interface (e.g., the communication interface (160) of FIG. 15).

[0106] In one example, the electronic device may receive a user input for selecting one of at least one generated clothing image. In one example, the electronic device may receive the user input via a user interface (e.g., the user interface (150) of FIG. 15). Alternatively, in one example, the electronic device may receive the user input from an external device (e.g., a user terminal or a server) via a communication interface.

[0107] According to the example described above, the electronic device can generate and provide multiple images related to the object the user wishes to register. By selecting and registering the desired image from among the multiple images, the user's satisfaction can be enhanced.

[0108] FIG. 8 is a flowchart illustrating a method for obtaining sound associated with an object according to one embodiment.

[0109] According to FIG. 8, according to one embodiment, the operating method may include an operation (S810) of obtaining at least one of context information corresponding to a user and category information of an object.

[0110] For example, if the electronic device is implemented as a clothing manager, the object may be clothing that the user is currently taking out or is about to take out from the clothing manager. Alternatively, the object may be clothing that the user is currently putting in or is about to put in the clothing manager. For example, context information corresponding to the user may include at least one of current weather information, the user's schedule information, the user's costume information, and the user's style information (or preferred style information). For example, the object's category information may include information about the type and color of the clothing.

[0111] For example, if the electronic device is implemented as a refrigerator, the object may be food that the user is currently taking out or is about to take out from the refrigerator. Alternatively, the object may be food that the user is currently putting in or is about to put in the refrigerator. In one example, the context information corresponding to the user may be context information related to food. In one example, the context information related to food may include information about various types of contexts related to food, including situations in which food is taken out of the refrigerator, situations in which food is put in the refrigerator, and situations in which food that was previously stored in the refrigerator is put back into the refrigerator.

[0112] In one example, an electronic device (e.g., the electronic device (100) of FIG. 2) may obtain context information and category information based on a user input. Alternatively, in one example, the electronic device may obtain at least one of the context information and the category information through an external device (e.g., a server or a user terminal).

[0113] According to one embodiment, the operating method may include an operation (S820) of acquiring a sound associated with an object based on at least one of acquired context information and category information. According to one example, the sound associated with the object may be a sound output through an electronic device while a user performs an action associated with the object, and may be a sound acquired based on the type of the object.

[0114] In one example, an electronic device can acquire sounds associated with an object using contextual information and category information. In one example, the act of "acquiring sounds" herein may include the act of "generating sounds" and the act of "identifying sounds."

[0115] For example, let's assume that an electronic device is implemented as a clothing manager. The electronic device can obtain contextual information corresponding to the user through a user interface (e.g., the user interface (150) of FIG. 15). For example, the electronic device can obtain "Today is a rainy day" as current weather information. The electronic device can obtain "Today is a workday" as the user's schedule information. The electronic device can obtain "Free Dress" as the user's clothing information. The electronic device can obtain "Preference for clothing in muted colors rather than usual colors" as the user's preferred style information.

[0116] For example, an electronic device can obtain category information of an object. For example, if the electronic device is implemented as a clothing manager, the object may be clothing currently selected by a user. The user can select any of the images registered as clothing images corresponding to the user, and the electronic device can identify the clothing image (or clothing) selected by the user. The electronic device can obtain category information of the clothing currently selected by the user based on information stored in a memory (e.g., memory (120) of FIG. 2). For example, the electronic device can obtain 'sky blue blouse' as category information based on a user input.

[0117] For example, an electronic device can acquire audio that reflects clothing category information along with contextual information, such as weather. For example, on a rainy day, audio containing the "sound of footsteps when walking in rain boots" may be acquired. Alternatively, for example, on a clear and windy day, when wearing thin clothing, audio containing the "audio generated when thin clothing sways in the wind on a windy day" may be acquired.

[0118] For example, if the object is food, the object's category information may be information about the type of food selected by the user. In another example, the electronic device may obtain category information about the food currently selected by the user based on information stored in memory. For example, the electronic device may obtain "cola" as category information based on user input.

[0119] For example, an electronic device may acquire a sound related to an object based on acquired context information and category information using a specified algorithm. For example, the electronic device may input the acquired context information and category information into a learned second neural network model to acquire a sound related to an object. For example, the learned second neural network model may be the same model as a first neural network model (e.g., the first neural network model of FIG. 2), but is not limited thereto, and the learned second neural network model may be configured separately from the first neural network model. For example, the learned second neural network model may be a model learned to output a sound related to an object when context information and category information of the object are input. The second neural network model will be described in detail with reference to FIGS. 13 and 14.

[0120] According to one embodiment, the operating method may include an operation (S830) of outputting the acquired sound through a speaker (e.g., speaker (170) of FIG. 15).

[0121] For example, an electronic device may output acquired sound through a speaker when a user's behavioral pattern related to an object is identified. For example, if the electronic device is implemented as a clothing manager, the electronic device may output acquired sound through the speaker when the user is present at a preset location (e.g., in front of the electronic device) or when the electronic device identifies the user as holding a garment at a preset location.

[0122] According to the example described above, sound related to an object can be output based on the user's context and the object's category. Accordingly, when a user uses an electronic device (e.g., a clothes dryer), sound related to the object selected or expected to be selected is played, thereby enhancing user satisfaction.

[0123] FIG. 9 is a flowchart illustrating a method of outputting acquired sound through a speaker according to one embodiment.

[0124] According to FIG. 9, according to one embodiment, the operating method may include an operation (S910) of obtaining a sound (e.g., a sound of FIG. 8) related to clothing selected by the user based on acquired context information (e.g., context information of FIG. 8) and category information of clothing selected by the user (e.g., category information of FIG. 8).

[0125] For example, an electronic device (e.g., the electronic device (100) of FIG. 2) may be implemented as a clothing manager. For example, the clothing selected by the user may be clothing that the user is currently taking out or plans to take out from the clothing manager. Alternatively, the clothing selected by the user may be clothing that the user is currently putting in or plans to put in the clothing manager.

[0126] For example, context information corresponding to a user may include at least one of current weather information, the user's schedule information, the user's clothing information, and the user's style information. For example, category information of an object may include information about the type and color of clothing. For example, an electronic device may obtain at least one of context information and category information based on a user input. Alternatively, the electronic device may obtain at least one of context information and category information from an external device (e.g., a server or a user terminal).

[0127] For example, the electronic device may input acquired context information and category information of clothing selected by the user into a learned second neural network model (e.g., the second neural network model of FIG. 8) to acquire sounds related to clothing selected by the user.

[0128] For example, an electronic device can obtain "Today is a clear and sunny day" as current weather information. The electronic device can obtain "Today is a day off" as user schedule information. The electronic device can obtain "Casual attire" as user clothing information. The electronic device can obtain "Prefers clothing in brighter colors than usual" as user style preference information. Upon receiving a user input selecting one of the clothing items stored in a clothing manager, the electronic device can identify "a yellow dress" as category information. The electronic device can input the obtained contextual information and category information into a trained second neural network model to obtain sounds including "the sound of birdsong and clothes made of bright materials rustling in the wind." In this way, the electronic device can obtain sounds by considering different types of information, including the texture of the clothing selected by the user and the current weather.

[0129] According to one embodiment, the operating method may include an operation (S920) of outputting the acquired sound through a speaker (e.g., speaker (170) of FIG. 15).

[0130] For example, if the electronic device is implemented as a clothes manager, the electronic device may output the acquired sound through the speaker when a specific trigger action is identified, such as when the user stands in front of the clothes manager or when the user stands in front of the clothes manager while holding clothes. Alternatively, for example, if the electronic device identifies a user's action (or movement) of taking clothes out of the electronic device, putting clothes into the electronic device, or selecting one of the clothes stored in the electronic device (or, a clothing image of a registered user (e.g., a clothing image corresponding to the user in FIG. 4A)), the electronic device may output the acquired sound through the speaker while the user performs the action.

[0131] For example, an electronic device can generate feedback based on a user's actions and perform actions related to the feedback. For example, if a user removes a garment from a garment care device while sound is being output through a speaker, the electronic device can provide information related to the garment selected by the user. In one example, the information related to the garment can include information about the number of times the garment has been worn or information about other garments worn with the selected garment, such as historical information about the garment.

[0132] For example, when a user takes out an item of clothing, an electronic device can provide information related to that item. For example, the electronic device can provide information such as, "This is the third time you've worn this item this year," or "You wore this item with white pants last time."

[0133] Alternatively, for example, when a user places clothing in the electronic device, the electronic device may recommend an operation course or a care method based on information related to the clothing. For example, the electronic device may provide the user with information such as, "Today's weather is humid, so we recommend the strong mode," or, "We recommend that wool coats operate in deodorizing mode," or, "We recommend that blouses be washed once." In one example, information related to clothing selected by the user may be stored in a memory (e.g., memory (120) of FIG. 2). Alternatively, information related to clothing may be stored in an external device (e.g., a server).

[0134] Fig. 10 is a flowchart illustrating a method of outputting acquired sound through a speaker according to one embodiment.

[0135] According to FIG. 10, according to one embodiment, the operating method may include an operation (S1010) of obtaining a sound (e.g., a sound of FIG. 8) related to the context of the food selected by the user, based on acquired context information (e.g., context information of FIG. 8) and category information of the food selected by the user (e.g., category information of FIG. 8).

[0136] For example, it may be assumed that an electronic device (e.g., the electronic device (100) of FIG. 2) is implemented as a refrigerator. For example, the object (e.g., the object of FIG. 2) may be food that a user is currently taking out or is about to take out from the refrigerator. Alternatively, the object may be food that a user is currently putting in or is about to put in the refrigerator. Alternatively, for example, the object may be food selected by the user. For example, the context information may be context information related to food. For example, the context information related to food may include information about different types of contexts related to food, including a situation in which food is taken out of the refrigerator, a situation in which food is put into the refrigerator, and a situation in which food that was previously stored in the refrigerator is put back into the refrigerator.

[0137] For example, the category information may be information about the type of food selected by the user. For example, the electronic device may obtain category information about the food currently selected by the user based on information stored in a memory (e.g., memory (120) of FIG. 2). For example, the electronic device may obtain 'cola' as the category information based on a user input. Alternatively, for example, the electronic device may obtain category information based on a captured image. For example, the electronic device may identify the type of object included in the captured image using a specified algorithm.

[0138] For example, an electronic device can input acquired contextual and category information into a trained second neural network model to acquire sounds associated with an object. For example, in a situation where a user takes a watermelon out of a refrigerator, the trained second neural network model can output the sound of "water flowing in a windy valley."

[0139] According to one embodiment, the operating method may include an operation (S1020) of outputting the acquired sound through a speaker (e.g., speaker (170) of FIG. 15).

[0140] For example, if the electronic device is implemented as a refrigerator, the electronic device may output the acquired sound through a speaker when a specific trigger action is identified, such as when a user stands in front of the refrigerator or when a user stands in front of the refrigerator while holding food.

[0141] FIG. 11 is a diagram for explaining a first neural network model (1100, e.g., the first neural network model of FIG. 2) according to one embodiment.

[0142] According to FIG. 11, according to one embodiment, the first neural network model (1100) may be a model trained to receive a captured image (1110, for example, the captured image of FIG. 2) and skeleton information (1120, for example, the skeleton information (421 and 422) of FIGS. 4A and 4B) corresponding to an object included in the captured image (1110), and output at least one clothing image (1130, for example, the clothing image of FIG. 8) for clothing included in the captured image (1110). According to one example, the skeleton information (1120) may include information on an occluded area (for example, the occluded areas (431, 432, and 433) of FIG. 4B) corresponding to an object included in the captured image (1110).

[0143] In one embodiment, the first neural network model (1100) may be a model implemented as a Generative Adversarial Network (GAN). In another example, the first neural network model (1100) may be a neural network model that uses a captured image (1110) and skeleton information (1120) corresponding to an object included in the captured image (1110) as learning data.

[0144] In one example, the first neural network model (1100) may output at least one clothing image (1130) of a different composition or a different viewpoint with respect to the object. In one example, the at least one clothing image (1130) output may be an image reflecting the texture of the clothing. In one example, the first neural network model (1100) may output an image of a different composition from the photographed image (1110). In one example, the at least one clothing image (1130) may be a 2D (Dimensional) image, but is not limited thereto, and may also be a 3D image.

[0145] For example, it can be assumed that the captured image (1110) includes an upper garment and a lower garment. In one example, an electronic device (e.g., the electronic device (100) of FIG. 2) can identify skeleton information (1120) corresponding to the upper garment from the captured image (1110). The skeleton information (1120) can include information about an occluded area corresponding to the upper garment. In one example, the electronic device can input the captured image (1110) and the skeleton information (1120) into a trained first neural network model (1100) to output upper garment images (1130) at different compositions and different viewpoints.

[0146] FIG. 12 is a diagram for explaining a first neural network model (1100, e.g., the first neural network model of FIG. 2) according to one embodiment.

[0147] According to FIG. 12, according to one embodiment, the first neural network model (1100) may be a model trained to receive a captured image (1110, for example, the captured image of FIG. 2) and skeleton information (1121, for example, the skeleton information (421 and 422) of FIGS. 4A and 4B) corresponding to an object included in the captured image (1110), and output at least one clothing image (for example, the clothing image of FIG. 8) for clothing included in the captured image (1110). According to one example, the skeleton information (1121) may include information on an occluded area (for example, the occluded areas (431, 432, and 433) of FIG. 4B) corresponding to an object included in the captured image (1110).

[0148] For example, it may be assumed that the captured image (1110) includes upper and lower garments. For example, an electronic device (e.g., the electronic device (100) of FIG. 2) may identify skeleton information (1121) corresponding to the lower garment from the captured image (1110). The skeleton information (1121) may include information about an occluded area corresponding to the lower garment. For example, the electronic device may input the captured image (1110) and the skeleton information (1121) into a trained first neural network model (1100) to output lower garment images (1131) at different compositions and different viewpoints.

[0149] For example, the learned first neural network model (1100) may output at least one clothing image corresponding to each of the upper garment and the lower garment. For example, the learned first neural network model (1100) may receive, together with the photographed image (1110), skeleton information corresponding to the upper garment (e.g., skeleton information (1120) of FIG. 11) and skeleton information (1121) corresponding to the lower garment, and output at least one clothing image corresponding to the upper garment (e.g., at least one clothing image (1130) of FIG. 11) and at least one clothing image (1131) corresponding to the lower garment.

[0150] FIG. 13 is a diagram for explaining a second neural network model (1300, e.g., the second neural network model of FIG. 8) according to one embodiment.

[0151] According to FIG. 13, according to one embodiment, the second neural network model (1300) may be a neural network model trained to input at least one of context information (1310, e.g., context information (1310) of FIG. 8) and category information (1320, e.g., category information (1320) of FIG. 8) corresponding to an object (e.g., an object of FIG. 2) and output a sound (1330, e.g., sound (1330) of FIG. 8) related to the object. According to one example, the second neural network model (1300) may be the same model as the first neural network model (e.g., the first neural network model (1100) of FIG. 11), but is not limited thereto, and the second neural network model (1300) may also be implemented as a different model from the first neural network model. As an example, the second neural network model may be implemented as a Generative Adversarial Network (GAN).

[0152] For example, if an electronic device (e.g., the electronic device (100) of FIG. 2) is implemented as a clothing manager, the second neural network model (1300) may receive context information (1310) corresponding to clothing and category information (1320) of clothing selected by the user, and output sound (1330) related to the clothing selected by the user. In this case, the sound (1330) related to the clothing selected by the user may be a sound (1330) that reflects the texture of the clothing and the current context (such as the current weather, the user's schedule, and the user's preferred style).

[0153] For example, if the electronic device is implemented as a refrigerator, the second neural network model (1300) may receive context information (1310) corresponding to food and category information (1320) of the food selected by the user, and output a sound (1330) related to the food selected by the user. In this case, the sound (1330) related to the food selected by the user may be a sound (1330) that reflects the category of the food and the current context (information about the current weather or whether the food is being taken out or put in).

[0154] FIG. 14 is a block diagram illustrating a learning method of a second neural network model (1300) according to one embodiment.

[0155] According to FIG. 14, according to one embodiment, the second neural network model (1300) can be trained to output sound (1390, e.g., sound (1330) of FIG. 13) using text generated based on context information (1310) and category information (1320) and audio (1380) related to the generated text as learning data.

[0156] According to one embodiment, an electronic device (e.g., the electronic device (100) of FIG. 2) may input context information (1310) and category information (1320) into a prompt generating module (1340). In one example, the prompt generating module (1340) may be a module that generates an input prompt (1350) for outputting a sound (1390) reflecting the context information (1310) and category information (1320). In one example, the input prompt (1350) may be a command (or text) input to a second neural network model.

[0157] For example, it can be assumed that an electronic device is implemented as a clothing manager. As an example, the electronic device can obtain 'Today is a rainy day' as current weather information. As the user's schedule information, the electronic device can obtain 'Today is a workday'. As the user's clothing information, the electronic device can obtain 'Free dress'. As the user's preferred style information, the electronic device can obtain 'Prefer clothes in calmer colors than usual colors'. As an example, the electronic device can obtain 'Sky blue blouse' as category information (1320). As an example, the electronic device can input the obtained information into the prompt generation module (1340) and obtain 'Sky blue blouse material raindrop sound' as the input prompt (1350).

[0158] According to one embodiment, the electronic device may input the acquired input prompt (1350) into a text encoding module (1360) to obtain an encoded input prompt.

[0159] According to one embodiment, the electronic device may input audio (1380) into the audio encoding module (1370) to obtain encoded audio information (or audio token). In one example, the audio (1380) may be sample audio corresponding to context information (1310) and category information (1320), which may be audio containing noise. For example, if the input prompt (1350) is 'raindrop sound on sky blue blouse material', the corresponding sample audio (1380) may be encoded.

[0160] According to one embodiment, the electronic device may perform learning by inputting an encoded prompt and encoded sample audio (1380) through a text encoding module (1360) into a second neural network model (1300). According to one example, the electronic device may perform learning by inputting an encoded prompt corresponding to 'raindrop sound of sky blue blouse material' and encoded sample audio (1380) corresponding to 'raindrop sound of sky blue blouse material' into the second neural network model (1300).

[0161] Fig. 15 is a block diagram showing a detailed configuration of an electronic device according to one embodiment.

[0162] According to FIG. 15, the electronic device (100') may include at least one processor (110), memory (120), display (130), at least one sensor (140), user interface (150), communication interface (160), speaker (170), and microphone (180). Among the configurations illustrated in FIG. 15, a detailed description of configurations that overlap with those illustrated in FIG. 2 will be omitted.

[0163] The display (130) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (130) may also include a driving circuit, a backlight unit, etc., which may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (130) may be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, etc. The processor (110) can control the display (130) to output an output image obtained according to the various embodiments described above. Here, the output image may be a high-resolution image of 4K or 8K or higher. The output image may also be a game image according to one embodiment.

[0164] According to one embodiment, the display (130) may include a plurality of haptic elements. The haptic elements may be implemented as motors for providing haptic feedback (e.g., vibration feedback) to a user, but are not limited thereto. According to one example, the display (130) may include a preset number of haptic elements. For example, the display (130) may include a preset number of haptic elements corresponding to a preset number of sub-regions of the display, but is not limited thereto, and it is to be understood that the display may include a different number of haptic elements than the number of sub-regions corresponding to the display.

[0165] At least one sensor (140, hereinafter referred to as a sensor) may include a plurality of sensors of various types. The sensor (140) may measure a physical quantity or detect an operating state of an electronic device (100') and convert the measured or detected information into an electrical signal. The sensor (140) may include a camera, and the camera may include a lens that focuses visible light or other optical signals reflected by an object and received onto an image sensor, and an image sensor that may detect visible light or other optical signals. Here, the image sensor may include a 2D pixel array divided into a plurality of pixels. Alternatively, at least one sensor (140) may include a temperature sensor or an infrared sensor.

[0166] The user interface (150) is a configuration for the electronic device (100') to interact with the user. For example, the user interface (150) may include, but is not limited to, at least one of a touch sensor, a motion sensor, a button, a jog dial, a switch, a microphone, or a speaker.

[0167] The communication interface (160) can input and output various types of data. For example, the communication interface (160) can transmit and receive various types of data to and from an external device (e.g., a source device), an external storage medium (e.g., a USB memory), an external server (e.g., a web hard drive) through a communication method such as AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc.

[0168] For example, the communication interface (160) may include a BLE (Bluetooth Low Energy) module. BLE refers to Bluetooth technology that enables transmission and reception of low-power, low-capacity data in the 2.4 GHz frequency band with a range of about 10 m. However, the present invention is not limited thereto, and the communication interface (160) may also include a Wi-Fi communication module. That is, the communication interface (160) may include at least one of a BLE (Bluetooth Low Energy) module or a Wi-Fi communication module.

[0169] According to one embodiment, the speaker (170) may be composed of a tweeter for reproducing high-frequency sounds, a midrange for reproducing mid-frequency sounds, a woofer for reproducing low-frequency sounds, a subwoofer for reproducing ultra-low-frequency sounds, an enclosure for controlling resonance, a crossover network for dividing the frequency of an electric signal input to the speaker into bands, etc.

[0170] According to one embodiment, the speaker (170) can output an audio signal to the outside of the electronic device (100'). The speaker (170) can output multimedia playback, recording playback, various notification sounds, voice messages, etc. The electronic device (100') may include an audio output device such as the speaker (170), but may also include an output device such as an audio output terminal. In particular, the speaker (170) can provide acquired information, information processed and produced based on acquired information, a response result to a user's voice, an operation result, etc. in the form of voice.

[0171] The microphone (180) may refer to a module that acquires sound and converts it into an electrical signal, and may be a condenser microphone, a ribbon microphone, a moving coil microphone, a piezoelectric element microphone, a carbon microphone, or a MEMS (Micro Electro Mechanical System) microphone. In addition, it may be implemented in an omnidirectional, bidirectional, unidirectional, sub-cardioid, super-cardioid, or hyper-cardioid manner. According to an embodiment, the electronic device (100') may include a microphone (180) and an inner microphone, and the microphone (180) may be a microphone that is positioned relatively outside the body. According to an example, the electronic device (100') may acquire an audio signal including external noise through the microphone (180). According to an embodiment, the microphone (180) may be placed in a direction opposite to the direction in which the speaker (170) emits sound.

[0172] According to the above-described example, the electronic device (100') can obtain an image that accurately expresses the texture and shape of the clothing that the user wishes to register by inputting the skeleton information of the clothing included in the captured image into a trained neural network model. In addition, the electronic device (100') can increase user convenience by obtaining an image of the clothing that includes the overall shape of the clothing even when a portion of the clothing included in the captured image is covered or cut off.

[0173] Meanwhile, the methods according to the various embodiments of the present disclosure described above may be implemented in the form of applications that can be installed on existing electronic devices. Alternatively, the methods according to the various embodiments of the present disclosure described above may be performed using a deep learning-based trained neural network (or a deep learned neural network), i.e., a learning network model. Furthermore, the methods according to the various embodiments of the present disclosure described above may be implemented only through a software upgrade or a hardware upgrade of an existing electronic device. Furthermore, the various embodiments of the present disclosure described above may also be performed through an embedded server provided in an electronic device or an external server of the electronic device.

[0174] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including commands stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The machine is a device that can call commands stored from the storage medium and operate according to the called commands, and may include a display device (e.g., display device (A)) according to the disclosed embodiments. When a command is executed by a processor, the processor can perform a function corresponding to the command directly or by using other components under the control of the processor. The command may include code provided or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0175] Furthermore, according to one embodiment, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily provided in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0176] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0177] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, At least one processor comprising a processing circuit; and A memory storing instructions and including one or more storage media; The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on the identification of the shooting trigger event, the shooting image is acquired, Identifying an object within the acquired captured image based on the acquired captured image, By inputting the acquired photographed image into the learned first neural network model and based on the acquired output data, an image corresponding to the identified object is generated, An electronic device that controls the above-mentioned generated image to be registered as content corresponding to a user.

2. In paragraph 1, The above instructions cause the electronic device to: Based on the above shooting trigger event being identified, a shooting image related to clothing is acquired, Based on the identification of an object in a photographed image related to the above clothing, skeleton information of the identified object is obtained, Generate a clothing image corresponding to the identified object based on the output data obtained by inputting at least one of the acquired skeleton information and the acquired photographed image into the learned first neural network model, An electronic device that controls the above-mentioned generated clothing image to be registered as a clothing image corresponding to a user.

3. In paragraph 2, The above instructions cause the electronic device to: Identify an occlusion area corresponding to the object based on the shape of the object, An electronic device that controls the generation of the clothing image based on inputting at least one of the skeleton information and the photographed image, and information about the identified occluded area into the learned first neural network model.

4. In paragraph 1, The above instructions cause the electronic device to: Based on the identification of the first object in the above-mentioned photographed image, first skeleton information of the identified first object is obtained, Based on the identification of the second object in the above-mentioned captured image, second skeleton information of the identified second object is obtained, An electronic device that controls to generate a first clothing image corresponding to the first object and a second clothing image corresponding to the second object, respectively, based on output data obtained by inputting the acquired first skeleton information, the acquired second skeleton information, and the acquired photographed image into the learned first neural network model.

5. In paragraph 1, The above instructions cause the electronic device to: An electronic device that provides a UI (User Interface) requesting re-capture of an image related to the object based on the identification of an occluded area corresponding to the object.

6. In paragraph 1, The above instructions cause the electronic device to: An electronic device that controls, based on a user input selecting at least one of the generated images, to register the selected one as an image corresponding to the user.

7. In paragraph 1, The above shooting trigger event is, An electronic device, which is an event that identifies the presence of a user or object at a preset location of the electronic device.

8. In paragraph 1, further comprising a communication interface; The above instructions cause the electronic device to: An electronic device that controls the acquisition of the photographed image from an external device through the communication interface.

9. In paragraph 1, Speaker; including more; The above instructions cause the electronic device to: Obtain at least one of context information corresponding to the user and category information of the object, Based on at least one of the acquired context information and the category information, acquire a sound related to the object, An electronic device that controls the acquired sound to be output through the speaker.

10. In paragraph 9, The above context information is, Contains at least one of information about the current weather, information about the user's schedule, and information about the user's use of the object; The above instructions cause the electronic device to: An electronic device that controls to acquire sound related to the object based on output data acquired by inputting the acquired context information and the acquired category information into a learned second neural network model.

11. In paragraph 2, The above instructions cause the electronic device to: An electronic device that controls sound related to clothing selected by a user based on a user input selecting one of the clothing images of the registered user.

12. In the method of controlling an electronic device A step of acquiring a shooting image based on the identification of a shooting trigger event; A step of identifying an object within the acquired photographed image based on the acquired photographed image; A step of generating an image corresponding to the identified object based on the acquired photographed image; and A method for controlling an electronic device, comprising: a step of registering the generated image as content corresponding to a user.

13. In the 12th paragraph, the method for controlling the electronic device is as follows: A step of acquiring a photographic image related to clothing based on the above photographing trigger event being identified; A step of obtaining skeleton information of the identified object based on the identification of the object in the photographed image related to the clothing; A step of generating a clothing image corresponding to the identified object based on at least one of the acquired skeleton information and the acquired photographed image; and A method for controlling an electronic device, comprising: a step of registering the generated clothing image as a clothing image corresponding to a user; 14. In the 12th paragraph, the control method of the electronic device, A step of obtaining at least one of context information corresponding to a user and category information of the object; A step of obtaining a sound related to the object based on at least one of the acquired context information and the category information; and A method for controlling an electronic device, comprising: a step of outputting the acquired sound; 15. A storage medium storing computer-readable instructions, wherein the instructions, when executed by at least one processor of an electronic device, cause the electronic device to: Based on the identification of the shooting trigger event, the shooting image is acquired, Identifying an object within the acquired captured image, Based on the acquired captured image, an image corresponding to the identified object is generated, A storage medium that causes the generated image to be registered as content corresponding to a user.

Images