User device and method for controlling same

Abstract

The present invention relates to a user device for reconfiguring home appliances in an indoor space and displaying an image for the reconfigured indoor space as a simulation image, and a method for controlling same. The present invention comprises: an image sensor for acquiring an indoor space image of an indoor space; a distance sensor for sensing the distances to a plurality of indoor objects in the indoor space; a processor for identifying elements of the indoor space on the basis of at least one of the indoor space image or the distances to the plurality of indoor objects, acquiring spatial information corresponding to the identified elements of the indoor space on the basis of the identified elements of the indoor space, and reconfiguring an interior design of the indoor space on the basis of the acquired spatial information; and a display unit for displaying the interior design of the indoor space as a simulated image.

Description

User device and control method thereof

[0001] The disclosed invention relates to a user device that recommends home appliances and a method for controlling the same.

[0002] Currently, many ordinary people living in residential spaces such as houses or apartments are attempting to change the interior and exterior of their living environments to suit their tastes or current trends.

[0003] Recently, interior design has expanded to include not only residential spaces but also indoor spaces of buildings such as offices, shops, hospitals, hotels, restaurants, cafes, department stores, beauty salons, clothing stores, factories, and schools, and refers to designing indoor spaces using interior finishing materials, furniture, lighting fixtures, home appliances, decorations, and curtains to create a pleasant indoor environment.

[0004] In other words, interior design refers to analyzing the lifestyles of ordinary people to design spaces that are more attractive and convenient, and to creating a pleasant indoor environment by materializing a wide variety of indoor spaces into appropriate functions and forms.

[0005] It is common for such interiors to be designed and constructed by specialized interior design firms. In this case, however, interior design firms often construct indoor spaces with uniform designs. Consequently, there is a problem in that users cannot receive interior services tailored to their tastes and personalities.

[0006] Even if a user presents an interior design that matches their personality and taste to an interior design company, users who are not majoring in interior design may only have a vague idea of ​​their preferred design and may not be able to explain it accurately.

[0007] When an indoor space is constructed with an interior design that does not reflect the user's intentions, there were inconvenient issues, such as having to repurchase or cancel at least one of the interior finishing materials, furniture, lighting, appliances, or decorations to change the interior design, as well as problems involving additional costs and time consumption due to the changes.

[0008] One aspect of the disclosed invention relates to a user device and a control method thereof that reconstructs a home appliance based on indoor space image information and distance information with indoor objects, and displays an image of the interior design of the reconstructed indoor space as a simulation image.

[0009] A user device according to one aspect includes: an image sensor for acquiring an indoor space image of an indoor space; a distance sensor for detecting distances to a plurality of indoor objects within the indoor space; a processor for identifying components of an indoor space based on at least one of the indoor space image and the distances to a plurality of indoor objects, acquiring spatial information corresponding to the identified components of the indoor space based on the identified components of the indoor space, and reconstructing an interior design of the indoor space based on the acquired spatial information; and a display unit for displaying the interior design of the indoor space as a simulated image.

[0010] A processor of a user device according to one aspect identifies at least one existing home appliance included in a plurality of indoor objects, obtains identification information of at least one existing home appliance based on the identified at least one existing home appliance, identifies operating specification information of at least one existing home appliance based on the obtained identification information of at least one existing home appliance, identifies an optimal location of at least one existing home appliance based on at least one of spatial information, identification information of at least one existing home appliance, and operating specification information of at least one existing home appliance, and reconstructs an interior design of an indoor space based on the identified optimal location.

[0011] A processor of a user device according to one aspect identifies at least one existing home appliance included in a plurality of indoor objects, and controls a display unit to display the identified at least one existing home appliance in a part of the display unit based on the identification of at least one existing home appliance included in a plurality of indoor objects.

[0012] A processor of a user device according to one aspect identifies boundary information of a plurality of indoor objects based on an indoor space image, identifies a first object including at least one existing home appliance and a second object other than the first object based on the identified boundary information of the plurality of indoor objects, and controls a display unit so that the first object and the second object are displayed in different ways in the indoor space image.

[0013] A processor of a user device according to one aspect identifies at least one existing home appliance included in a plurality of indoor objects, obtains identification information of at least one existing home appliance based on the identified at least one existing home appliance, identifies at least one recommended home appliance among at least one new home appliance that corresponds to an indoor space based on the obtained identification information of at least one existing home appliance and identification information of at least one new home appliance stored in advance, and controls a display unit to display the identification information of at least one recommended home appliance identified.

[0014] A processor of a user device according to one aspect identifies at least one recommended appliance based on whether at least one new appliance is of the same type as or interacts with at least one existing appliance.

[0015] A processor of a user device according to one aspect identifies the optimal location of at least one recommended home appliance based on identification information and spatial information of at least one recommended home appliance, and reconstructs the interior design of an indoor space based on the optimal location of the identified at least one recommended home appliance.

[0016] A user device according to one aspect further includes an input unit that receives user input. A processor of a user device according to one aspect reconstructs the interior design of an indoor space based on the selection of at least one recommended home appliance from the input unit.

[0017] According to one aspect, spatial information of a user device includes information about a first object including at least one existing home appliance existing within an indoor space, information about a second object other than the first object, and psychological element information about the first and second objects. According to one aspect, psychological element information of a user device includes at least one of brand information, shape information, size information, material information, and color information of the first and second objects.

[0018] A user device according to one aspect further includes an input unit that receives priority information regarding the reconfiguration of the interior design of an indoor space. A processor of a user device according to one aspect determines the ranking of at least one recommended home appliance based on the priority information received through the input unit, indoor space information, and identification information of at least one existing home appliance present in the indoor space, and controls a display unit to display identification information of at least one recommended home appliance based on the determined ranking. The priority information of a user device according to one aspect includes at least one information among the interior harmony, price, space utilization, and energy efficiency of the home appliance.

[0019] A processor of a user device according to one aspect determines a ranking between at least one existing home appliance and at least one recommended home appliance based on priority information, spatial information, and identification information of at least one existing home appliance received through an input unit.

[0020] A control method for a user device according to another aspect comprises: acquiring an indoor space image of an indoor space; identifying distances to a plurality of indoor objects within the indoor space; identifying components of the indoor space based on at least one of the indoor space image and the distances to the plurality of indoor objects; acquiring spatial information corresponding to the identified components of the indoor space based on the identified components of the indoor space; reconstructing an interior design of the indoor space based on the acquired spatial information; and displaying the interior design of the indoor space as a simulation image.

[0021] The step of reconstructing the interior design of an indoor space includes: identifying at least one existing home appliance included in a plurality of indoor objects; obtaining identification information of at least one existing home appliance based on the identified at least one existing home appliance; identifying operation specification information of the existing home appliance based on the obtained identification information of at least one existing home appliance; identifying the optimal location of at least one existing home appliance based on at least one of spatial information, identification information of at least one existing home appliance, and operation specification information of at least one existing home appliance; and reconstructing the interior design of the indoor space based on the optimal location of the identified at least one existing home appliance.

[0022] The step of reconstructing the interior design of an indoor space includes: identifying at least one existing home appliance included in a plurality of indoor objects; obtaining identification information of at least one existing home appliance based on the identified at least one existing home appliance; and identifying at least one recommended home appliance among at least one new home appliance that corresponds to the indoor space based on the obtained identification information of at least one existing home appliance and identification information of at least one new home appliance stored in advance.

[0023] The step of identifying at least one recommended appliance includes identifying at least one recommended appliance based on whether at least one new appliance is of the same type as or interacts with at least one existing appliance.

[0024] According to the disclosed invention, the disclosed invention can improve the user's perception of the reconstruction of home appliances by reconstructing home appliances within an indoor space based on environmental element information of the indoor space and simulating an image of the reconstructed indoor space.

[0025] The disclosed invention can provide a new interior design that reflects the user's style, personality, and preferences, rather than a standardized interior design, by recommending new home appliances based on information on environmental elements of an indoor space and priorities regarding the reconfiguration conditions of home appliances, or by providing placement locations for existing home appliances and recommended home appliances, and can improve user satisfaction with interior design services.

[0026] The disclosed invention generates multiple scenarios corresponding to the placement locations of existing home appliances or recommended home appliances, and displays simulation images in augmented reality for each scenario, thereby allowing users to experience interior designs of various indoor spaces in advance, and thereby enabling users to easily rearrange home appliances or purchase home appliances.

[0027] The disclosed invention allows for placing home appliances in a location where energy efficiency is maximized, thereby enabling the maximum utilization of the appliance's performance while reducing the amount of energy used within the indoor space and extending the appliance's lifespan.

[0028] The disclosed invention allows a home appliance selected by a user to be placed at a location selected by the user within an indoor space through augmented reality, and by displaying the interior design of the indoor space where the home appliance is placed as a simulated image, it enables the user to easily recognize the interior design of the indoor space before actually placing the home appliance, thereby improving user convenience.

[0029] The disclosed invention allows users to receive recommendations for home appliances suitable for the interior design of their indoor space and to visualize the placement of the recommended appliances within the indoor space through augmented reality. Through this, the disclosed invention can prevent placement errors of home appliances within the indoor space or disharmony between the indoor space and the appliances, and can minimize inconveniences such as the exchange or refund of purchased appliances.

[0030] For a company that manufactures and sells home appliances, the disclosed invention may enable the proposal of a new advertising model capable of providing promotional information to users.

[0031] The disclosed invention can improve the marketability of home appliances and further secure the competitiveness of home appliances.

[0032] FIG. 1 is a configuration diagram of an interior design service system including a user device according to an embodiment.

[0033] FIG. 2 is a control configuration diagram of a user device according to an embodiment.

[0034] Figure 3 is an example image of a home structure displayed on a user device according to an embodiment.

[0035] Figure 4 is an example of a scan of an indoor space of a user device according to an embodiment.

[0036] FIG. 5 is an example diagram of distance detection between an indoor object and a user device in an indoor space using a user device according to an embodiment.

[0037] FIG. 6 is an example of image processing of a user device according to an embodiment.

[0038] FIGS. 7 and 8 are example diagrams of the recognition of a home appliance by a user device according to an embodiment.

[0039] FIGS. 9A and 9B are example diagrams of boundary recognition of indoor objects through a user device according to an embodiment.

[0040] FIG. 10 is an example of material recognition of indoor objects through a user device according to an embodiment.

[0041] FIGS. 11 to 19 are example diagrams of the display of a simulation image of a user device according to an embodiment.

[0042] FIG. 20 is a control configuration diagram of a user device according to an embodiment.

[0043] FIG. 21 is a control flowchart for recommending and displaying home appliances of a user device according to an embodiment.

[0044] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0045] In relation to the description of the drawings, similar reference numerals may be used for similar or related components.

[0046] The singular form of the noun corresponding to an item may include one or plural items, unless the relevant context clearly indicates otherwise.

[0047] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0048] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish a component from another component and do not limit the components in other aspects (e.g., importance or order).

[0049] Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that the component may be connected to the other component directly (e.g., via a wire), wirelessly, or through a third component.

[0050] Terms such as “include” or “have” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this document, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0051] When it is said that a component is "connected," "combined," "supported," or "in contact" with another component, this includes not only cases where the components are directly connected, combined, supported, or in contact, but also cases where they are indirectly connected, combined, supported, or in contact through a third component.

[0052] When it is said that a component is located "on" another component, this includes not only cases where one component is in contact with the other, but also cases where another component exists between the two components.

[0053] The term “and / or” includes a combination of multiple related described components or any of the multiple related described components.

[0054] The operating principle and embodiments of the present invention will be described below with reference to the attached drawings.

[0055] FIG. 1 is an exemplary diagram of an interior design service system including a user device according to an embodiment of the present disclosure.

[0056] The interior design service system (1) can recognize environmental element information of the indoor space based on indoor space information of the indoor space, reconfigure home appliances within the indoor space based on the recognized environmental element information of the indoor space, and provide an interior design service with the reconfigured home appliances to the user.

[0057] The reconfiguration of home appliances within an indoor space may include at least one of the rearrangement of existing home appliances, the recommendation of new home appliances, and the placement of new home appliances.

[0058] Hereinafter, home appliances existing in the indoor space will be listed as 'existing home appliances,' and new home appliances that may be newly added to the indoor space will be listed as 'recommended home appliances.'

[0059] Indoor space information may include image information of the indoor space and distance information between indoor objects within the indoor space and the user device.

[0060] Information on environmental elements of an indoor space may further include information on indoor objects and psychological elements.

[0061] Indoor objects can include static and dynamic elements.

[0062] Static elements may include at least one of a ceiling, wall, floor surface, stairs, door, window, and electrical outlet.

[0063] Dynamic elements may include home appliances and may further include furniture, lighting, and decorative items.

[0064] Psychological element information may include information on the material, shape, and color of static elements, and information on the material, shape, and color of dynamic elements.

[0065] The interior design service system (1) recognizes the user's intention regarding the current interior design based on information regarding environmental elements of the indoor space, and it is also possible to reconfigure home appliances based on the recognized user's intention.

[0066] The user's intention regarding interior design may include the atmosphere of the indoor space and the user's style regarding the current interior design.

[0067] The atmosphere of an indoor space may include a warm atmosphere, a cold atmosphere, a bright atmosphere, a dark atmosphere, a cozy atmosphere, a heavy atmosphere, a sophisticated atmosphere, a bright atmosphere, an urban atmosphere, and a glamorous atmosphere, and examples of the atmosphere of an indoor space are not limited to these.

[0068] The user's style may include at least one of modern style, mid-century modern style, Nordic Scandinavian style, industrial style, bohemian style, cozy casual style, rustic style, and glamorous style, but the user's style is not limited thereto.

[0069] The interior design service system (1) recognizes the suitability of placement of existing home appliances based on environmental element information of the indoor space and can provide information about the suitability of placement of existing home appliances to the user.

[0070] Information regarding the placement suitability of home appliances may include information on the placement suitability of existing home appliances and information on the placement unsuitability of existing home appliances.

[0071] The interior design service system (1) can recognize that the relocation of the home appliance is necessary based on information regarding the unsuitability of the placement of the existing home appliance.

[0072] When the interior design service system (1) recognizes that the existing home appliance needs to be relocated, it can recognize location information of the existing home appliance based on environmental element information of the indoor space and provide the user with relocation information of the home appliance corresponding to the recognized location information.

[0073] The interior design service system (1) can recommend new home appliances suitable for the indoor space to the user based on information about environmental elements of the recognized indoor space.

[0074] Recommending new home appliances to a user may include recommending home appliances of a different type or model than those already present in the indoor space, or recommending additional home appliances needed for the indoor space.

[0075] The interior design service system (1) can provide the user with information on the placement of recommended home appliances.

[0076] The interior design service system (1) can display interior design images using augmented reality technology and image processing technology.

[0077] The interior design video may include at least one of an indoor space video containing an image of an existing home appliance placed at a first location, an indoor space video containing an image of an existing home appliance placed at a second location, an indoor space video containing an image of a recommended home appliance at the first location, and an indoor space video containing an image of a recommended home appliance at a third location.

[0078] The interior design service system (1) can provide an interior design video of an indoor space based on augmented reality as a simulation video.

[0079] The interior design service system (1) can provide an interior design video of an indoor space based on augmented reality as a three-dimensional indoor space video.

[0080] The interior design service system (1) may include a user device (100) and a server (200), and may further include a home appliance (300).

[0081] The user device (100) can be carried by the user or placed in the user's home or office, etc.

[0082] The user device (100) can be implemented as a computer or portable terminal that can connect to an external device through a network.

[0083] Here, a computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser, but is not limited thereto.

[0084] A portable terminal is a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0085] The user device (100) may include a communication module capable of communicating with a server (200) or a home appliance (300), a user interface that receives user input or outputs information to the user, at least one processor that controls the operation of the user device (100), and at least one memory in which a program for controlling the operation of the user device (100) is stored. The specific configuration of the user device (100) will be described later.

[0086] A program, i.e., an application, for providing an interior design service for an indoor space can be stored in the memory of the user device (100). The application may be sold with the user device (100) installed, or may be downloaded from a server and installed.

[0087] In addition, it is possible to store a program, i.e., an application, for controlling the home appliance (300) in the memory of the user device (100).

[0088] By running an application installed on the user device (100), the user can access the server (200) to create a user account and communicate with the server (200) based on the logged-in user account.

[0089] A user device (100) can register a home appliance (300) through a server (200). For example, if the home appliance (300) is operated to connect to the server (200) according to the procedure guided by an application installed on the user device (100), the home appliance (300) can be registered to the user account by the server (200) registering the identification information of the home appliance (300) (e.g., serial number or MAC address, etc.) to the user account.

[0090] The user can control the home appliance (300) using an application installed on the user device (100). For example, when the user logs into a user account using an application installed on the user device (100), the home appliance (300) registered to the user account appears, and when the user inputs a control command for the home appliance (300), the control command can be transmitted to the home appliance (300) through the server (200).

[0091] The server (200) can be implemented as various computing devices such as a workstation, cloud, data drive, and data station.

[0092] The server (200) may be implemented as one or more servers that are physically or logically separated based on functions, detailed configurations of functions, or data, and may transmit and receive data and process the transmitted and received data through communication between each server.

[0093] The server (200) may include a communication module capable of communicating with another server, a home appliance (300) or a user device (100), at least one processor capable of processing data received from another server, a home appliance (300) or a user device (100), and at least one memory capable of storing a program for processing data or processed data.

[0094] The server (200) stores information about a user device (100) and a plurality of home appliances, and can transmit the stored information about the plurality of home appliances to the user device (100).

[0095] Information regarding multiple home appliances may include identification information, appearance information, and operation specification information for each of the multiple home appliances.

[0096] The identification information of a home appliance may include at least one of the name information, model information, and serial information of the home appliance.

[0097] The appearance information of a home appliance may include at least one of the size information, color information, border information, material information, and shape information of the home appliance.

[0098] The operating specification information of a home appliance may include type information of the home appliance, structural information of the home appliance, performance information of the home appliance, space size information regarding the space where the home appliance can be placed, optimal location information of the home appliance where the performance of the home appliance is maximized, and unsuitable location information where the performance of the home appliance is degraded.

[0099] For example, if the home appliance is an air purifier, the operating specification information of the air purifier may include location information of the intake port, location information of the discharge port, the number of discharge directions (1-way, 2-way, 3-way, 4-way, etc.), size information of the indoor space corresponding to the air purification capacity, optimal location information where the air purification capacity is maximized (around a door or window), unsuitable location information (around a bed), and optimal location information to maximize energy efficiency.

[0100] As another example, if the home appliance is a television, the television's operating specification information may include viewing distance information corresponding to the size of the television.

[0101] As another example, if the home appliance is an air conditioner, the operating specification information of the air conditioner may include information on the size of the indoor space corresponding to the cooling and heating capacity of the air conditioner, and may further include information on the optimal location to maximize energy efficiency.

[0102] As another example, if the home appliance is a washing machine, the operating specification information of the washing machine may include optimal location information (location information of water pipes and faucets).

[0103] The server (200) can further store style information of interior designs corresponding to each of the multiple home appliances.

[0104] The server (200) can perform functions such as managing user accounts, registering user devices (100) and home appliances (300) associated with user accounts, and managing or controlling the registered user devices (100) and home appliances (300). For example, a user can create a user account by accessing the server (200) through a user device (100). A user account can be identified by an ID and password set by the user.

[0105] The server (200) can register the user device (100) and the home appliance (300) to the user account according to a set procedure. For example, the server (200) can register, manage, and control the home appliance (300) by linking the identification information of the home appliance (300) (e.g., serial number or MAC address, etc.) to the user account.

[0106] When the server (200) receives information regarding operation or status from the home appliance (300), it updates the stored information regarding operation or status of the home appliance (300) and transmits the updated information regarding operation and status of the home appliance (300) to the user device (100) via a network. Here, updating information may include various operations that change existing information, such as adding new information to existing information or replacing existing information with new information.

[0107] The interior design service system (1) may include a home network system that establishes an environment in which home appliances (300) within the home communicate with each other through a network and can be interconnected via the internet between a user device (100) and at least one home appliance (300).

[0108] A network can include both wired and wireless networks.

[0109] Wired networks include cable networks or telephone networks, etc., and wireless networks may include all networks that transmit and receive signals via radio waves. Wired networks and wireless networks may be connected to each other.

[0110] Networks may include wide area networks (WANs) such as the Internet, local area networks (LANs) formed around access points (APs), and short-range wireless networks that do not use access points (APs). Short-range wireless networks may include, but are not limited to, Bluetooth (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc.

[0111] An access point (AP) can connect a home appliance (300) or a user device (100) to a wide area network (WAN) to which a server (200) is connected. The home appliance (300) or the user device (100) can be connected to the server (200) through the wide area network (WAN).

[0112] The access point (AP) can communicate with a home appliance (300) or a user device (100) using wireless communication such as Wi-Fi (IEEE 802.11), Bluetooth (IEEE 802.15.1), or Zigbee (IEEE 802.15.4), and can connect to a wide area network (WAN) using wired communication, but is not limited thereto.

[0113] According to various embodiments, the home appliance (300) may be directly connected to the user device (100) or server (200) without going through the access relay (AP).

[0114] The home appliance (300) may be at least one of various types of home appliances. For example, the home appliance (300) may include at least one of a refrigerator, dishwasher, electric range, electric oven, air conditioner, garment care machine, washing machine, dryer, microwave oven, fan, and air purifier, but is not limited thereto. For example, it may include various types of home appliances such as a cleaning robot, vacuum cleaner, humidifier, dehumidifier, water purifier, and television.

[0115] The home appliance (300) may include a communication module capable of communicating with other home appliances, a user device (100), or a server (200), a user interface that receives user input or outputs information to the user, at least one processor that controls the operation of the home appliance (300), and at least one memory in which a program for controlling the operation of the home appliance (300) is stored.

[0116] The home appliance (300) can transmit information regarding operation or status to other home appliances, user devices (100), or servers (200) through a network.

[0117] The home appliance (300) may operate according to control commands received from other home appliances, user devices (100), or servers (200). For example, if the home appliance (300) has obtained prior approval from a user to operate according to control commands from servers (200) even without user input, the home appliance (300) may operate according to control commands received from servers (200). Here, the control commands received from servers (200) may include, but are not limited to, control commands entered by the user through user devices (100) or control commands based on pre-set conditions.

[0118] A home appliance (300), a user device (100), or a server (200) may determine a control command using technology such as artificial intelligence. For example, the server (200) may receive information regarding the home appliance (300) or information regarding the user of the user device (100), process it using technology such as artificial intelligence, and transmit the processing result or a control command to the home appliance (300) or the user device (100) based on the processing result.

[0119] FIG. 2 is a control configuration diagram of a user device according to an embodiment, which will be explained with reference to FIGS. 3 to 19.

[0120] The user device (100) may include a user interface (110), an image sensor (120), a distance sensor (130), a communication unit (140), and a control unit (150).

[0121] The user device (100) may include a main body forming an exterior. A user interface (110), an image sensor (120), a distance sensor (130), a communication unit (140), and a control unit (150) may be provided in the main body.

[0122] The user interface (110) can receive user input and output various information regarding the operation of the user device (100).

[0123] The user interface (110) may include an input section (111), a display section (112), and a speaker (113).

[0124] The input unit (111) receives user input and transmits the received user input to the control unit (150).

[0125] The input unit (111) can receive execution and termination commands for an application for interior design services, and can receive scan commands for an indoor space and display commands for a simulated image.

[0126] The input unit (111) can receive identification information of multiple indoor spaces within the home and can receive selection information of an indoor space among the multiple indoor spaces to receive interior design services.

[0127] The input unit (111) can receive selection information of an existing home appliance or selection information of at least one recommended home appliance related to an existing home appliance.

[0128] The input unit (111) can receive priority information regarding the priority of the reconfiguration conditions of the home appliance.

[0129] The home appliance reconfiguration conditions are for reconfiguring existing or recommended home appliances within an indoor space, and may be conditions for receiving recommendations for the placement location of each home appliance or for receiving recommendations for new home appliances.

[0130] Reconfiguration conditions may include space utilization, interior harmony, and energy efficiency, and may additionally include price.

[0131] That is, the input unit (111) can receive priority information regarding the reconfiguration conditions for each home appliance.

[0132] For example, if the home appliance is an air conditioner, energy efficiency can be received as the first priority of the reconfiguration conditions for the air conditioner, space utilization can be received as the second priority, and interior harmony can be received as the third priority.

[0133] As another example, if the home appliance is a television, space utilization can be received as the first priority of the television's reconfiguration conditions, interior harmony can be received as the second priority, and space utilization can be received as the third priority.

[0134] The input unit (111) may also receive selection information for one of the multiple home appliances displayed on the display unit (112) and placement information for one of the home appliances from the user.

[0135] Here, the plurality of home appliances may include multiple home appliances of different types. For example, the plurality of home appliances may include an air conditioner, an air purifier, and a television.

[0136] Multiple home appliances may include multiple home appliances of the same type but different models. For example, the multiple home appliances may include a first model air conditioner, a second model air conditioner, and a third model air conditioner.

[0137] The placement information for home appliances may include location information where the home appliances are to be placed.

[0138] The input unit (111) can receive selection information of a home appliance within an indoor space image displayed on the display unit (112) and can receive a rotation command of the home appliance.

[0139] When the input unit (111) is implemented as a touch pad, the input unit (111) can receive touch signals corresponding to tap, double tap, press, pan, swipe, flick, pinch, drag, and drag and drop.

[0140] In addition to buttons and jog dials, the input unit (111) may include hardware devices such as switches, pedals, keyboards, mice, trackballs, various levers, handles, or sticks.

[0141] The input unit (111) may further include a microphone for receiving the user's voice.

[0142] The input unit (111) may include a device that is a GUI (Graphical User Interface), such as a touch pad, i.e., software. The touch pad may be implemented as a touch screen panel (TSP) and form a layered structure with the display unit (112).

[0143] The display unit (112) displays the output information of the user device (100) as an image based on the control command of the control unit (150).

[0144] The display unit (112) can display the screen of an application for interior design services.

[0145] The display unit (112) can display an image of the indoor space where the home appliance is placed as an augmented reality image.

[0146] The display unit (112) can also display information about the recognized user's style or the atmosphere of the indoor space.

[0147] When displaying an augmented reality image, the display unit (112) can display identification information of existing home appliances placed in an indoor space and display an image of the exterior of existing home appliances placed in an indoor space.

[0148] When displaying an augmented reality image, the display unit (112) can display identification information of the recommended home appliance, display an image of the exterior of the recommended home appliance, and further display price information and sales location information of the recommended home appliance.

[0149] The display unit (112) displays list information of home appliances placed in an indoor space, and it is also possible to display an image of the exterior of the home appliance as list information.

[0150] The display unit (112) can display an augmented reality image of the current interior design of the indoor space as a simulation image, and can display an augmented reality image of the new interior design of the indoor space as a simulation image.

[0151] The display unit (112) can display simulation images according to scenario.

[0152] For example, the scenario may include a scenario in which an existing home appliance is placed at a first location, a scenario in which an existing home appliance is placed at a second location, a scenario in which a recommended home appliance is placed at a first location, a scenario in which a recommended home appliance is placed at a second location, and a scenario in which a recommended home appliance is placed at a third location.

[0153] The display unit (112) can display a simulation image of an augmented reality image of a new interior design in which a home appliance selected by the user is placed at a location selected by the user.

[0154] That is, the display unit (112) can display an augmented reality image of a new interior design created automatically or manually as a simulation image.

[0155] The display unit (112) can display a rotated image of a home appliance within an indoor space image based on a rotation command of the home appliance received through the input unit (111). For example, the display unit (112) can rotate and display the image of the home appliance up to 360 degrees.

[0156] The display unit (112) can display priority information regarding the reconfiguration conditions of the home appliance for each home appliance.

[0157] The display unit (112) can display home structure information, can display structure information for each room within the home, and can also display list information of existing home appliances arranged in each room.

[0158] The display unit (112) can display an image of an indoor space for an interior design service, and can also display information about the boundary of the indoor space and information about environmental elements of the indoor space.

[0159] The display unit (112) includes a plurality of seven-segment displays.

[0160] The display unit (112) may be provided with a liquid crystal display (LCD), a digital light processing (DLP) panel, a plasma display panel, an electroluminescence (EL) panel, an electrophoretic display (EPD) panel, an electrochromic display (ECD) panel, a light-emitting diode (LED) panel, or an organic light-emitting diode (OLED) panel, but is not limited thereto.

[0161] The speaker (113) can output output information of the user device (100) as sound based on a control command of the control unit (150).

[0162] The image sensor (120) acquires an indoor space image of the indoor space and transmits indoor space image information for the acquired indoor space image to the control unit (150).

[0163] The image sensor (120) may include a camera.

[0164] The image sensor (120) may include a CCD or CMOS image sensor.

[0165] The camera may also include three-dimensional spatial recognition cameras such as KINECT (RGB-D sensor), TOF (Time of Flight) camera, and stereo camera.

[0166] The distance sensor (130) can detect the distance to indoor objects provided in the indoor space.

[0167] The distance sensor (130) can detect the distance between the distance sensor (130) and the indoor objects based on a reference position in the indoor space.

[0168] When detecting the distance to indoor objects, a user device (100) may be provided at a reference location in the indoor space. In this case, the distance sensor (130) of the user device provided at the reference location in the indoor space can detect the distance between the distance sensor (130) and the indoor objects over a part of the indoor space or the entire area (360 degrees).

[0169] Indoor objects may include a first object including a home appliance, and a second object including at least one of a wall, floor surface, ceiling, stairs, electrical outlet, furniture, and decoration of the indoor space.

[0170] For example, the distance sensor (130) can detect the distance between the distance sensor (130) and at least one wall of the indoor space, the distance between the distance sensor (130) and the ceiling of the indoor space, the distance between the distance sensor (130) and the floor of the indoor space, the distance between the distance sensor (130) and a home appliance (300), and the distance between the distance sensor (130) and furniture.

[0171] The distance sensor (130) may be one or more.

[0172] Two or more distance sensors can be installed vertically on the user device.

[0173] Two or more distance sensors can be provided on the left and right sides of the user device.

[0174] The distance sensor (130) can be rotated left and right or up and down, and it is also possible to change the field of view for distance detection through rotation.

[0175] The distance information with indoor objects detected by the distance sensor (130) can be used to recognize the size and shape of the indoor space or to recognize the size and shape of environmental elements of the indoor space.

[0176] Here, the environmental elements of the indoor space are indoor objects, which may include static and dynamic elements.

[0177] The distance sensor (130) may include at least one of an optical sensor, a lidar sensor, an ultrasonic sensor, or a radar sensor.

[0178] The distance sensor (130) can emit at least one of infrared, ultrasonic, or radio waves, and can receive at least one of infrared, ultrasonic, or radio waves reflected from an indoor object.

[0179] The distance sensor (130) can detect the distance between the distance sensor (130) and an indoor object based on the intensity of at least one of the received infrared, ultrasonic, or radio waves. For example, the distance sensor (130) can detect the distance between the distance sensor (130) and an indoor object based on the difference (or phase difference) between the time of at least one of the infrared, ultrasonic, or radio waves being transmitted and the time of at least one of the infrared, ultrasonic, or radio waves being received.

[0180] The communication unit (140) may include various communication circuits for performing wired communication and / or wireless communication with external devices (e.g., servers, other user devices and / or home appliances).

[0181] The communication unit (140) may include at least one of a short-range communication circuit and a long-range communication circuit.

[0182] The communication unit (140) can transmit data to an external device or receive data from an external device. For example, the communication unit (140) may support cellular communication, wireless local area network, home radio frequency, infrared communication, ultra-wide band communication, Wi-Fi, Wi-Fi Direct, Bluetooth, AD-HOC, and / or Zigbee. The communication technologies supported by the communication unit (140) are not limited to those exemplified.

[0183] The communication unit (140) may also communicate with an external device through an access point (AP).

[0184] The communication unit (140) may receive information about a home appliance from the server (200) in response to a control command from the control unit (150) and transmit the received information about the home appliance to the control unit (150).

[0185] Information about the home appliance may include identification information of the home appliance, appearance information of the home appliance, and operation specification information of the home appliance.

[0186] The communication unit (140) communicates with the home appliance (300) in response to a control command from the control unit (150), receives information about the home appliance from the home appliance (300), and can also transmit the received information about the home appliance to the control unit (150).

[0187] The control unit (150) can be electrically connected to various parts and / or devices of the user device (100) and can control various parts and / or devices. That is, the control unit (150) can control the overall operation of the user device (100).

[0188] The control unit (150) can control the operation of the user device (110) based on user input received by the input unit (111).

[0189] The control unit (150) can control at least one of the display unit (112) and the speaker (113) so that output information related to the operation of the user device (100) is output.

[0190] Application Run Configuration

[0191] The control unit (150) controls the execution of the application based on the reception of an execution command of the application through the input unit (111), terminates the execution of the application based on the reception of an termination command of the application through the input unit (111), and can control at least one of the display unit (112) and the speaker (113) to output execution information of the application during the execution of the application.

[0192] Here, the application may include an application for interior design services for indoor spaces.

[0193] The control unit (150) can store identification information of an indoor space for receiving an interior design service received through the input unit (111) during the execution of the application.

[0194] The control unit (150) can store structural information of the home received from the server (200), and it is also possible to store identification information of a plurality of indoor spaces of the home based on user input received through the input unit (111). The user input received through the input unit (111) may include selection information of an indoor space and identification information of a selected indoor space.

[0195] The control unit (150) can control the display unit (112) to display structural information of the home.

[0196] The control unit (150) may also receive size information and shape information of each indoor space from the server (200). For example, the server (200) may store home structure information for a shared residential space such as an apartment.

[0197] The control unit (150) receives indoor space image information for each of the multiple indoor spaces within the home acquired by the image sensor (120), and can store the indoor space image information for each received indoor space by matching it with the identification information of the indoor space.

[0198] For example, identification information for indoor spaces may include Room 1, Room 2, Room n, etc. N may be a natural number. As another example, identification information for indoor spaces may include a living room, bathroom, bedroom, kitchen, and utility room, etc.

[0199] As illustrated in FIG. 3, when the control unit (150) controls the display unit (112) to display the identification information of the indoor space and the indoor space image information for each indoor space of the home by matching the identification information of the indoor space and the indoor space image information with the structural information of the home.

[0200] <Indoor Space Scan Configuration>

[0201] The control unit (150) can recognize identification information of an indoor space among a plurality of indoor spaces to receive interior design services based on user input received through the input unit (111).

[0202] The control unit (150) can control the activation of the image sensor (120) and the distance sensor (130) based on a scan command of the indoor space received through the input unit (111).

[0203] The control unit (150) can receive indoor space image information obtained by the activated image sensor (120) and distance information with indoor objects obtained by the activated distance sensor (130).

[0204] The indoor objects include a first object including a home appliance, and a second object including static and dynamic elements excluding the home appliance.

[0205] Static elements may include at least one of a ceiling, wall, floor surface, stairs, door, and window.

[0206] Dynamic elements may include furniture, lighting, and decorative items. Additionally, the first object, the home appliance, may be included in the dynamic elements.

[0207] The indoor space image information and the distance information to indoor objects may be information acquired at the same time.

[0208] During the same time period, the field of view of the image sensor (120) when acquiring indoor space image information may be the same as the field of view of the distance sensor (130) when acquiring distance information with indoor objects.

[0209] When the time of acquisition of indoor space image information acquired by the image sensor (120) and the time of acquisition of distance information between indoor objects acquired by the distance sensor (130) are different, the control unit (150) recognizes first objects in the indoor space image information, recognizes the shape and size of the recognized first objects, recognizes the shape and size of second objects based on the distance information between indoor objects, recognizes second objects identical to each of the first objects by comparing the shape and size of the first objects and the shape and size of the second objects, and it is also possible to recognize distance information between first objects included in the indoor space image information based on the matching information between the first objects and the second objects.

[0210] The control unit (150) can control the vertical rotation of the image sensor (120) to acquire images of the entire area of ​​the indoor space, both vertically and horizontally, and can also control the horizontal rotation of the image sensor (120).

[0211] The control unit (150) may also control at least one of the display unit (112) and the speaker (113) to output movement guidance information for the user device to acquire an image of the entire area of ​​the indoor space.

[0212] As illustrated in FIG. 4, the control unit (150) may also control the user interface (110) to output guidance information for a panoramic scan so that an image of the entire area of ​​the indoor space can be acquired in a panoramic manner.

[0213] The control unit (150) can control the distance sensor (130) to detect the distance to indoor objects vertically from the floor to the ceiling of the indoor space, and to detect the distance to indoor objects horizontally from the first wall to the second wall. Here, the first wall and the second wall may be walls facing each other.

[0214] The control unit (150) can also control the up and down and left and right rotation of the distance sensor (130).

[0215] When a plurality of distance sensors are arranged vertically, the control unit (150) can also control the left and right rotation of the distance sensor (130).

[0216] When multiple distance sensors are provided arranged on the left and right sides, the control unit (150) can also control the vertical rotation of the distance sensor (130).

[0217] <Environmental Element Recognition Configuration>

[0218] The control unit (150) can recognize environmental element information of the indoor space based on distance information detected by the distance sensor (130).

[0219] The control unit (150) can recognize the size information and shape information of the indoor space based on the distance information with indoor objects received from the distance sensor (130).

[0220] The control unit (150) can recognize the location information, size information, and shape information of indoor objects based on the distance information of indoor objects received from the distance sensor (130).

[0221] As illustrated in FIG. 5, the control unit (150) recognizes a wall (W) adjacent to a recognized indoor object among the walls of an indoor space based on distance information with indoor objects received from a distance sensor (130), and can recognize a first width information (D1), a second width information (D2), and a third width information (D3) of a recognized indoor object based on distance information between the recognized wall (W) and the distance sensor (130) and distance information between the recognized indoor object and the distance sensor (130) according to distance detection positions based on the transmission and reception of signals from the distance sensor (130).

[0222] For example, the control unit (150) can recognize size information of an indoor object based on the first width information (D1), second width information (D2), and third width information (D3) of the recognized indoor object.

[0223] The control unit (150) can also recognize shape information of an indoor object based on distance information recognized by distance detection location according to the transmission and reception of a signal from the distance sensor (130).

[0224] Shape information of indoor objects may include information corresponding to the boundaries of indoor objects, and shape information for straight lines and curves.

[0225] Indoor objects whose distance is detected by a distance sensor (130) may include a first object and a second object.

[0226] The control unit (150) can recognize an empty area in an indoor space based on the size information of the indoor space, the shape information of the indoor space, the location information of indoor objects, the size information of indoor objects, and the shape information of indoor objects, and can recognize the size information and shape information of the recognized empty area. Here, the empty area may be an area in the indoor space where no indoor object is placed.

[0227] The control unit (150) can recognize environmental element information of the indoor space based on the appearance information of indoor objects stored in the memory (152) and indoor space image information obtained by the image sensor (120).

[0228] Here, the environmental elements of the indoor space may include indoor objects that can be provided in the indoor space. The indoor objects that can be provided in the indoor space may include static elements and dynamic elements.

[0229] Indoor objects that may be provided in an indoor space may include a first object including home appliances and a second object including dynamic and static elements excluding home appliances. That is, the first object may include various types of home appliances. The second object may include at least one of a wall, ceiling, floor surface, stairs, window, door, furniture, decoration, and lighting.

[0230] The control unit (150) can recognize environmental element information of the indoor space using artificial intelligence.

[0231] The control unit (150) recognizes an empty area in the indoor space based on indoor space image information acquired by the image sensor (120), and based on distance information with indoor objects, it is also possible to recognize the size information and shape information of the recognized empty area based on space size information, shape information of the indoor space, location information of indoor objects, size information of indoor objects, and shape information of indoor objects.

[0232] As illustrated in FIG. 6, the control unit (150) processes indoor space image information and recognizes pixels (EP) corresponding to the edges in the processed indoor space image, recognizes a plurality of indoor objects based on the recognized pixels (EP), and can recognize edge information of indoor objects for each indoor object based on the recognized pixels (EP).

[0233] The control unit (150) recognizes a first object among a plurality of recognized indoor objects based on the boundary information of the indoor object for each recognized indoor object and the boundary information of the first objects stored in the memory (152), and can recognize identification information of the recognized first object and operation specification information of the first object based on the boundary information of the recognized first object and the boundary information of the first objects stored in the memory (152).

[0234] The identification information of the first object may include name information of the home appliance, model information of the home appliance, and serial information of the home appliance. The name information of the first object may be information corresponding to the type of home appliance, such as a refrigerator, washing machine, air conditioner, etc.

[0235] When the first object is a refrigerator, the refrigerator operation specification information may include refrigerator type information. The refrigerator type information may include type information corresponding to the arrangement of the door and storage compartment, such as French Door Type, Side-by-side Type, BMF (Bottom Mounted Freezer), TMF (Top Mounted Freezer), or 1-door refrigerator type.

[0236] When the first object is an air conditioner, the type information of the air conditioner may include a ceiling type, a stand type, and a wall-mounted type depending on the placement location, and may include a 4-way type, a 1-way type, and a duct type depending on the air discharge method.

[0237] The control unit (150) can control the display unit (112) to display identification information of existing home appliances placed in the indoor space.

[0238] As illustrated in FIG. 7, the control unit (150) can control the display unit (112) to display the image of the first object placed in the indoor space and the image of the second object placed in the indoor space differently from each other.

[0239] As illustrated in FIG. 8, the control unit (150) can control the user interface (110) to display an indoor space image in a first area (m1) of the interface (110) and control the user interface (110) to display identification information of a first object in a second area (m2) of the user interface (110).

[0240] Controlling the user interface may include controlling the display unit (112).

[0241] The control unit (150) may also control the user interface (110) to display the appearance information of the first object in the second area (m2) of the user interface (110).

[0242] The control unit (150) can also recognize at least one home appliance in the indoor space image information through artificial intelligence and learning of the exterior information of multiple home appliances and indoor space image information stored in the memory (152).

[0243] When multiple home appliances are provided in an indoor space, the control unit (150) can control the display unit (112) to generate a list of recognized home appliances and display list information of home appliances for the generated list in the second area (m2).

[0244] As illustrated in FIGS. 9a and 9b, the control unit (150) can recognize identification information of indoor objects and appearance information of second objects for each second object based on the boundary information of indoor objects for each recognized indoor object and the boundary information of second objects stored in memory (152).

[0245] That is, the control unit (150) can recognize identification information and appearance information of indoor objects excluding home appliances.

[0246] The control unit (150) can exclude the border information of home appliances from the border information of indoor objects for each recognized indoor object, and it is also possible to recognize only the border information of indoor objects excluding the border information of home appliances.

[0247] The control unit (150) can recognize a representative shape of the indoor space based on the boundary information of the indoor space and the boundary information of the indoor objects.

[0248] The control unit (150) can recognize a representative shape of the indoor space based on the shape information of the indoor space and the shape information of the indoor objects.

[0249] The control unit (150) recognizes the number of curved indoor objects and the number of straight indoor objects based on the shape information of the indoor objects, and can recognize the representative shape of the indoor space based on the number of curved indoor objects and the number of straight indoor objects.

[0250] The control unit (150) can recognize the representative shape of the indoor space as a curved shape based on the fact that the number of indoor objects with a curved shape is greater than or equal to a reference number, and recognize the representative shape of the indoor space as a straight shape based on the fact that the number of indoor objects with a straight shape is greater than or equal to a reference number.

[0251] The control unit (150) can recognize the representative shape of the indoor space as a curved shape based on the fact that the number of curved indoor objects is greater than the number of straight indoor objects, and can recognize the representative shape of the indoor space as a straight shape based on the fact that the number of straight indoor objects is greater than the number of curved indoor objects.

[0252] The control unit (150) can recognize operation specification information of the recognized first object based on indoor space image information, distance information with the first object and distance information with the second object, and can recognize usage guidance information of the second object.

[0253] For example, the operation specification information of the first object may include guidance information regarding the opening and closing directions of the door of the home appliance.

[0254] For example, the usage guide information of the second object may include guidance information regarding the furniture's doors, windows, and the opening and closing directions of the doors.

[0255] The control unit (150) can recognize color information and material information of indoor objects in the indoor space based on indoor space image information.

[0256] The control unit (150) can recognize the main color and auxiliary color of the indoor space based on indoor space image information and a pre-stored reference color table.

[0257] A standard color chart is used to serve as a reference point for colors and is called a color bar, color chip, or Pantone color chip.

[0258] The dominant color can be the color that occupies the largest proportion in the image of an indoor space.

[0259] The secondary color can be the color that accounts for the second-largest proportion in an indoor space image.

[0260] The control unit (150) can recognize reflection information of a plurality of indoor objects recognized based on indoor space image information, and recognize material information of each of the plurality of indoor objects based on the reflection information of the recognized plurality of indoor objects.

[0261] Material information may include information regarding glossy and matte materials.

[0262] The control unit (150) can recognize the color distribution and clarity of a plurality of indoor objects recognized based on indoor space image information, and can also recognize the material information of each of the plurality of indoor objects based on the color distribution and clarity of the recognized plurality of indoor objects.

[0263] The control unit (150) can store material information of each of the plurality of indoor objects by matching it with an image of each of the plurality of indoor objects.

[0264] As illustrated in FIG. 10, the control unit (150) recognizes the number of indoor objects made of a glossy material and the number of indoor objects made of a matte material, compares the recognized number of indoor objects made of a glossy material and the number of indoor objects made of a matte material, recognizes the material with the larger number, and recognizes the recognized material as the representative material of the indoor space.

[0265] When the control unit (150) receives identification information of a home appliance from a home appliance (300) placed in an indoor space, it transmits request information requesting appearance information of the home appliance to the server (200), and when the appearance information of the home appliance is received from the server (200), it is also possible to match the received appearance information of the home appliance with the identification information of the home appliance and store it.

[0266] The appearance information of the home appliance includes shape information, size information, color information, and material information of the home appliance corresponding to the identification information of the home appliance, and may further include edge information of the home appliance.

[0267] The control unit (150) may also receive operation specification information of the home appliance from the server (200).

[0268] User Style Awareness

[0269] The control unit (150) can also recognize the user's style for the current interior design based on psychological element information.

[0270] Psychological element information may include shape information, size information, color information, material information, and brand information of the first object, and shape information, size information, color information, material information, and brand information of the second object.

[0271] For example, the control unit (150) can recognize the user's style as Nordic Scandinavian style based on the recognition of wooden furniture, bright colored walls, and natural-colored decorations.

[0272] As another example, the control unit (150) can recognize the user's style as industrial style based on the recognition of a wall made of bricks with a primitive feel, iron furniture, and decorations with a rough texture.

[0273] As another example, the control unit (150) can recognize the user's style as bohemian style based on the recognition of colorful accessories such as layered rugs, colorful cushions, and fabric art.

[0274] As another example, the control unit (150) can recognize the user's style as a glamorous style based on the recognition of furniture made of materials such as velvet, silk, and fur, curtains and cushions made of elegant and cozy materials, and crystal chandeliers and metal ornaments.

[0275] <Recommended Home Appliance Configurations>

[0276] The control unit (150) can control the user interface (110) to recognize a new home appliance based on the identification information of an existing home appliance and the user's style, and to display the recognized home appliance as a recommended home appliance.

[0277] The memory of a server or user device can store style information for interior designs by home appliance.

[0278] The control unit (150) can control the user interface (110) to recognize a new home appliance based on the representative shape of the indoor space and to display the recognized home appliance as a recommended home appliance.

[0279] The control unit (150) can recognize a new home appliance based on identification information of home appliances stored in the server (200) or memory (152).

[0280] The control unit (150) can control the user interface (110) to recognize a new home appliance based on identification information of existing home appliances, the main color and auxiliary color of the indoor space, and to display the recognized home appliance as a recommended home appliance.

[0281] The new home appliance may be of the same type as the existing home appliance, or it may be of a different type that is interrelated with the existing home appliance.

[0282] For example, if the existing home appliance is an air conditioner, another type of home appliance related to the air conditioner may be an air purifier.

[0283] As another example, if the existing home appliance is a television, another type of home appliance related to the television could be an audio system.

[0284] As illustrated in FIG. 11, the control unit (150) can control the user interface (110) to display an indoor space image in a first area (m1) of the user interface (110) and control the user interface (110) to display identification information of recommended home appliances in a second area (m2) of the user interface (110).

[0285] When there are multiple recommended home appliances, the control unit (150) can control the user interface (110) to display identification information of multiple recommended home appliances as list information in the second area (m2).

[0286] When the control unit (150) controls the display of recommended home appliances in the second area (m2) of the user interface (110), it can control the user interface (110) to display specification information and price information for each recommended home appliance together in the recommendation area (m2) and to display an indoor space image in the first area of ​​the user interface (110).

[0287] The control unit (150) may also control the user interface (110) to display recommended home appliances and existing home appliances together in the second recommendation area (m2).

[0288] The control unit (150) can primarily recognize a home appliance of the same type as an existing home appliance among a plurality of home appliances stored in the server (200) based on identification information of a plurality of home appliances stored in the server (200), secondarily recognize a home appliance having a main color or an auxiliary color among the home appliances primarily recognized, and recognize the home appliances secondarily recognized as recommended home appliances.

[0289] The control unit (150) can recognize a new home appliance based on the representative material of the indoor space and identification information of existing home appliances, and recognize the recognized home appliance as a recommended home appliance for recommendation.

[0290] The control unit (150) can primarily recognize a home appliance of the same type as an existing home appliance among a plurality of home appliances stored in the server (200) based on identification information of a plurality of home appliances stored in the server (200), secondarily recognize a home appliance of a representative material among the home appliances primarily recognized, and recognize the home appliances secondarily recognized as recommended home appliances.

[0291] The control unit (150) can recognize a new home appliance based on the size information of the indoor space and the identification information of the existing home appliance, and it is also possible to recognize the recognized home appliance as a recommended home appliance.

[0292] The control unit (150) can recognize energy efficiency information of a home appliance corresponding to an indoor space and recognize a new home appliance based on the recognized energy efficiency information.

[0293] For example, if the home appliance is an air conditioner, the control unit (150) recognizes heating and cooling capacity information based on the size information of the indoor space, and can recognize the air conditioner with the recognized heating and cooling capacity information as a recommended air conditioner.

[0294] As another example, when the home appliance is an air purifier, the control unit (150) recognizes balance information based on the size information of the indoor space and can recognize the air purifier with the recognized balance information as a recommended air purifier.

[0295] The control unit (150) can recognize size information of a home appliance corresponding to size information of an indoor space and recognize a new home appliance based on the recognized size information of the home appliance.

[0296] For example, when the home appliance is a television, the control unit (150) recognizes viewing distance information based on the size information of the indoor space, recognizes the size information of the television based on the recognized viewing distance information, and can recognize the television having the recognized size information as a recommended television.

[0297] <Simulation Configuration>

[0298] The control unit (150) can control the user interface (110) to display an indoor space image as an augmented reality image based on a display command of a simulation image received through the input unit (111).

[0299] That is, the control unit (150) can control the user interface (110) to display a simulation image of the indoor space as an augmented reality image.

[0300] The control unit (150) can control the user interface (110) to generate a three-dimensional indoor space image based on indoor space image information and distance information with indoor objects, and to display the generated three-dimensional indoor space image as a simulation image.

[0301] The control unit (150) can control the user interface (110) so that when selection information and rotation information of a home appliance are received through the input unit (111) while a simulation image is displayed through the user interface (110), it recognizes identification information of the home appliance based on the received selection information of the home appliance and displays an image of the recognized home appliance while rotating based on the received rotation information.

[0302] The control unit (150) can control the user interface (110) to display a three-dimensional indoor space image of the current interior design as a simulation image.

[0303] The control unit (150) can control the user interface (110) to display a three-dimensional indoor space image of the reconstructed interior design as a simulation image.

[0304] <Simulation Configuration After Placement / Recommendation>

[0305] The control unit (150) recognizes the suitability of placement of existing home appliances based on environmental element information of the indoor space, identification information of existing home appliances, location information of existing home appliances, and operation specification information of existing home appliances, and recognizes that relocation of home appliances is necessary based on information on unsuitability of placement of existing home appliances.

[0306] When the control unit (150) recognizes that the existing home appliance needs to be relocated, it recognizes the optimal location information of the existing home appliance based on the environmental element information of the indoor space and the operation specification information of the existing home appliance, generates new indoor space image information based on the recognized optimal location information and the image information of the existing home appliance, and controls the user interface (110) to output the new indoor space image information.

[0307] For example, if the existing home appliance is a 4-way air purifier, the control unit (150) recognizes whether the distance between the air purifier and the wall is less than a reference distance based on distance information detected by a distance sensor, and recognizes that the air purifier needs to be relocated based on the recognition that the distance between the air purifier and the wall is less than the reference distance.

[0308] The control unit (150) can recognize areas of the indoor space where the distance from the wall is greater than or equal to a reference distance, and among the recognized areas, an empty area can be recognized as the optimal location for relocating the air purifier.

[0309] When generating new indoor space image information, the control unit (150) generates indoor space image information in the first stage by deleting image information of existing home appliances from indoor space image information acquired by the image sensor (120), generates indoor space image information in the second stage based on the indoor space image information generated in the first stage, image information of existing home appliances, and recognized optimal location information, and can generate indoor space image information in the second stage as three-dimensional indoor space image information.

[0310] As illustrated in FIG. 12, when the control unit (150) receives selection information for an existing home appliance from the input unit (111) while identification information for a plurality of existing home appliances is displayed in the second area of ​​the user interface, it can recognize identification information for an existing home appliance for relocation based on the selection information for an existing home appliance received from the input unit (111), recognize optimal location information for an existing home appliance based on the recognized identification information for an existing home appliance, environmental element information of the indoor space, and operation specification information of the existing home appliance, and generate new indoor space image information based on the recognized optimal location information and image information of the existing home appliance.

[0311] Here, the new indoor space image information may include indoor space image information in which the home appliance has been reconstructed, and may include three-dimensional indoor space image information.

[0312] As illustrated in FIG. 13, the control unit (150) may recognize existing home appliances of the same type as the recommended home appliance based on identification information of the recommended home appliance, recognize location information of the recognized existing home appliance, generate new indoor space image information based on the recognized location information, identification information of the recommended home appliance, image information of the recommended home appliance, and indoor space image information, and control the user interface (110) to display the generated new indoor space image information in the first area (m1).

[0313] More specifically, the control unit (150) can generate indoor space image information in the first step by deleting image information of existing home appliances from indoor space image information obtained by the image sensor (120), and generate indoor space image information in the second step by adding image information of recommended home appliances to the indoor space image information generated in the first step.

[0314] The control unit (150) can generate new indoor space image information by adding an image of a recommended home appliance to a location where an existing home appliance is placed.

[0315] The control unit (150) may recognize the optimal location information of the recommended home appliance based on identification information of the recommended home appliance, operation specification information of the recommended home appliance, and environmental element information of the indoor space, generate new indoor space image information based on the recognized optimal location information, image information of the recommended home appliance, and indoor space image information, and control the user interface (110) to display the generated new indoor space image information in the first area (m1).

[0316] As illustrated in FIG. 14, when the control unit (150) receives selection information for a recommended home appliance from the input unit (111) while identification information for a plurality of recommended home appliances is displayed in the second area (m1) of the user interface, it is possible to control the user interface (110) to recognize identification information for a recommended home appliance based on the selection information for a recommended home appliance received from the input unit (111), recognize optimal location information for a recommended home appliance based on the recognized identification information for a recommended home appliance, operation specification information for a recognized recommended home appliance, and environmental element information of an indoor space, generate new indoor space image information based on the recognized optimal location information, identification information for a recommended home appliance, and indoor space image information, and display the generated new indoor space image information in the first area (m1).

[0317] Here, the new indoor space image information may include three-dimensional indoor space image information.

[0318] When the control unit (150) receives selection information for recommended home appliances and location information for recommended home appliances from the input unit (111) while identification information for multiple recommended home appliances is displayed in the second area (m1) of the user interface, it is also possible to generate new indoor space image information by adding an image of a recommended home appliance to an indoor space image based on the received location information of the recommended home appliances.

[0319] The control unit (150) can recognize a first empty area of ​​the indoor space based on the size information of the indoor space, the shape information of the indoor space, the size information of the indoor objects, the shape information of the indoor objects, and the location information of the indoor objects, delete the image of the home appliance from the indoor space image, and recognize a second empty area in the indoor space image from which the home appliance has been deleted. That is, the control unit (150) can recognize the location information of the second empty area, the shape information of the second empty area, and the size information of the second empty area based on the shape information, location information, and size information of the home appliance.

[0320] The first empty area may be an area in the indoor space where indoor objects are not provided.

[0321] The second empty area may be an area within the indoor space where home appliances are provided.

[0322] The control unit (150) can recognize a capacitor in an indoor space image and recognize location information of the recognized capacitor.

[0323] The control unit (150) can recognize a possible placement area for a recommended home appliance based on information of a first empty area, information of a second empty area, location information of a capacitor, and appearance information of a recommended home appliance.

[0324] The control unit (150) can recognize an optimal area among the possible placement areas based on the operation specification information of the recommended home appliance. Here, the optimal area may be the optimal location where the recommended home appliance is to be placed.

[0325] The existing home appliance is a washing machine (301), and the configuration of the control unit when recommending a washing machine of a different model from the existing washing machine model as a recommended washing machine is described.

[0326] As illustrated in FIG. 15, the control unit (150) recognizes the position information, size information, and shape information of the washing machine (301) based on at least one of the indoor space image information obtained by the image sensor (120) and the distance information with indoor objects detected by the distance sensor (130), and recognizes the position information, size information, and shape information of the second object (Box, Sh).

[0327] The control unit (150) can recognize a first empty area of ​​the indoor space based on the size information of the indoor space, the shape information of the indoor space, the size information of the indoor objects, the shape information of the indoor objects, and the location information of the indoor objects, delete the image of the existing washing machine from the indoor space image, and recognize a second empty area in the indoor space image from which the existing washing machine has been deleted.

[0328] The control unit (150) recommends a new washing machine based on information of the first empty area, information of the second empty area, and location information of the second object, and can recognize whether the recommended washing machine can be placed based on information of the first empty area, information of the second empty area, location information of the second object, and appearance information of the recommended washing machine.

[0329] The control unit (150) may recommend a top-loading washing machine based on the location information of the second object (Sh) and recommend one of the front-loading washing machines based on the second object (Sh) being located above the existing washing machine in the indoor space.

[0330] The control unit (150) can recognize the optimal area among the possible placement areas of the recommended washing machine based on the operation specification information and appearance information of the recommended washing machine of the front-loading method, the location information of the second object (Sh), the information of the first empty area, and the information of the second empty area.

[0331] As illustrated in FIG. 16, the control unit (150) can delete the image of the second object (Box) of the dynamic element from the indoor space image and recognize the third empty area in the indoor space image from which the second object (Box) of the dynamic element has been deleted. That is, the control unit (150) can recognize the location information of the third empty area, the shape information of the third empty area, and the size information (L1, L2) of the third empty area based on the shape information, location information, and size information of the second object of the dynamic element and the distance information to the wall (W).

[0332] The second object (Box) of the dynamic elements may include objects among the dynamic elements excluding home appliances.

[0333] The third empty area may be an area in the interior space of dynamic elements (e.g., furniture, lighting, household goods, and decorations, etc.) excluding home appliances where interior objects are not provided.

[0334] The wall (W) may be a wall adjacent to the rear of the second object of the dynamic element in the interior space.

[0335] The control unit (150) recognizes a placement area of ​​a recommended home appliance based on information of a first empty area, information of a second empty area, location information of a capacitor, information of a third empty area, and appearance information of a recommended home appliance, and it is also possible to recognize an optimal area among the placement areas based on operation specification information of a recommended home appliance.

[0336] The control unit (150) may also control the user interface (120) to recognize identification information of a dynamic element placed in the third empty area based on the recognition that the optimal area is the third empty area, and to display guidance information for the movement guidance of the dynamic element and the placement guidance of the recommended home appliance based on the identification information of the recognized dynamic element.

[0337] The guidance information regarding the movement guidance of dynamic elements and the placement guidance of recommended home appliances may include information indicating that recommended home appliances can be placed in an optimal area when the dynamic elements are moved to a different location.

[0338] The control unit (150) recognizes whether the recommended home appliance can be placed based on information of the first empty area, information of the second empty area, location information of the capacitor, and appearance information of the recommended home appliance, and based on the recognition that the recommended home appliance cannot be placed in the indoor space, it deletes the image of dynamic elements excluding the home appliance from the indoor space image and recognizes the third empty area in the indoor space image from which the dynamic elements have been deleted.

[0339] This explains the case where the existing home appliance is a washing machine (301) and the recommended home appliance is a dryer.

[0340] As illustrated in FIG. 16, the control unit (150) deletes the image of the laundry box (Box) from the indoor space image, recognizes a third empty area in the indoor space image from which the image of the laundry box has been removed, and can recognize size information (L1, L2) and shape information of the third empty area based on the distance information from the laundry box and the distance information from the wall (W) detected by the distance sensor (130).

[0341] The wall (W) may be a wall adjacent to the rear of the laundry box in the indoor space.

[0342] The control unit (150) can recognize whether a dryer can be placed in the third empty area based on the size information (L1, L2) and shape information of the third empty area and the size information and shape information of the dryer, which is a recommended home appliance.

[0343] As illustrated in FIG. 17, the control unit (150) can control the user interface (110) so that an indoor space image is displayed in the first area (m1) of the user interface based on the recognition that a dryer can be placed in the third empty area, and control the user interface (110) so that an image of a recommended home appliance is displayed in the second area (m2) of the user interface.

[0344] The control unit (150) may also control the user interface (120) to display guidance information regarding the movement guidance of the laundry box (Box) and the placement guidance of the dryer in the third area (m3) of the user interface based on the recognition that the dryer can be placed in the third empty area.

[0345] The control unit (150) can control the user interface (120) to display a simulation image in which a dryer is placed in a third empty area based on receiving selection information for one of a plurality of recommended dryers through the input unit (111).

[0346] The control unit (150) may generate multiple scenarios based on placement information for each of the multiple recommended home appliances, placement information for existing home appliances, and relocation information for existing home appliances, and control the user interface (120) to display a simulation video corresponding to each of the generated multiple scenarios.

[0347] For example, multiple scenarios may include a scenario in which an existing home appliance is placed at a first location, a scenario in which an existing home appliance is placed at a second location, a scenario in which a recommended home appliance is placed at a first location, a scenario in which a recommended home appliance is placed at a second location, and a scenario in which a recommended home appliance is placed at a third location.

[0348] The control unit (150) may recognize an electrical outlet, window, wall, gas pipe, faucet, etc. among indoor objects, recognize whether a recommended home appliance can be placed based on location information of at least one of the recognized electrical outlet, window, wall, gas pipe, faucet, and the user interface (120) may also control the user interface to display location information where a recommended home appliance can be placed based on the recognition that the placement of the recommended home appliance is possible.

[0349] The control unit (150) can display a simulation video as an augmented reality video for each scenario, thereby allowing the user to experience the interior design of various indoor spaces in advance, and thereby enable the user to easily rearrange or purchase home appliances.

[0350] Priority Control

[0351] The control unit (150) can receive priority information regarding the priority of the reconstruction conditions of home appliances in the indoor space received by the input unit (111), and can reconstruct home appliances in the indoor space based on the received priority information.

[0352] The conditions for reconfiguring home appliances within an indoor space are conditions for reconfiguring existing or recommended home appliances within the indoor space, and may be conditions for receiving recommendations for the placement locations of home appliances or for receiving recommendations for new home appliances.

[0353] Reconfiguration conditions may include space utilization, interior harmony, and energy efficiency, and may additionally include price.

[0354] The control unit (150) can receive one or more priority information for each type of home appliance through the input unit (111).

[0355] <Priority Control - Priority One>

[0356] Below, the configuration of the control unit is described in the case where one priority information is received for each type of home appliance.

[0357] One priority may be the highest priority and should be listed as the first priority. Explain the configuration of the control unit when recommending a home appliance of the same type as an existing home appliance.

[0358] The control unit (150) receives selection information of an existing home appliance received through the input unit (111), and recognizes identification information of an existing home appliance based on the received selection information of the existing home appliance.

[0359] The control unit (150) recognizes a first priority selected by the user based on priority information received through the input unit (111).

[0360] The control unit (150) recognizes identification information of home appliances of the same type as existing home appliances based on identification information of multiple new home appliances and identification information of recognized existing home appliances. Hereinafter, home appliances of the same type as existing home appliances are listed as candidate home appliances.

[0361] The identification information of multiple new home appliances may be the identification information of home appliances stored in the memory (152) of the server (200) or the user device.

[0362] The control unit (150) can recognize one or more recommended home appliances among a plurality of candidate home appliances based on the recognized first priority. This will be explained in more detail.

[0363] The control unit (150) recognizes the appearance information of each candidate home appliance based on the recognized first priority being interior harmony, and recognizes the similarity between the appearance of the candidate home appliance and the environmental element of the indoor space based on the recognized appearance information of each candidate home appliance and the environmental element information of the indoor space.

[0364] The appearance information of each candidate home appliance may include information regarding the shape, color, and material of the candidate home appliance.

[0365] Information on environmental elements of an indoor space may include information on psychological elements. Information on psychological elements may include information on the shape, color, and material of the indoor space, and information on the shape, color, material, and brand of indoor objects.

[0366] Information on environmental elements of an indoor space may include information on representative materials, representative shapes, primary colors, and secondary colors.

[0367] The control unit (150) can recognize the similarity between the information regarding the shape, color, and material of each candidate home appliance and the information regarding the representative material, representative shape, main color, and auxiliary color, and determine the ranking of each candidate home appliance based on the recognized similarity.

[0368] The control unit (150) can recognize one or more candidate home appliances as recommended home appliances, starting from the first rank among the rankings of candidate home appliances.

[0369] The control unit (150) can recognize one or more candidate home appliances that have a similarity greater than or equal to a reference similarity among the similarities of candidate home appliances as recommended home appliances.

[0370] The control unit (150) assigns a first score to a candidate home appliance that corresponds to the same as the representative material, assigns a second score that corresponds to the same as the representative shape, assigns a third score that corresponds to the same as the main color, assigns a fourth score that corresponds to the same as the auxiliary color, sums the first, second, third, and fourth scores for each candidate home appliance, determines the ranking of each candidate home appliance based on the total score for each candidate home appliance, and can recognize one or more candidate home appliances as recommended home appliances, starting from the first ranking among the rankings of the candidate home appliances and pre-set rankings.

[0371] The control unit (150) recognizes the size information of a placeable area where a home appliance can be placed based on the size information of an empty area and the size information of a candidate home appliance based on the recognized first priority being space utilization, and recognizes candidate home appliances that can be placed in the placeable area as recommended home appliances based on the size information of the recognized placeable area and the size information of the candidate home appliance.

[0372] The control unit (150) can recognize the empty area as a placeable area based on the size information of the empty area and the size information of the candidate home appliance, and based on the fact that the size of the empty area is larger than the size of the candidate home appliance.

[0373] When multiple recommended home appliances are recognized, the control unit (150) recognizes the size difference between the placeable area and the candidate home appliances based on the size information of the recognized placeable area and the size information of the candidate home appliances, and it is also possible to determine the ranking of the candidate home appliances based on the recognized size difference and the standard size difference.

[0374] The control unit (150) can also determine the ranking of candidate home appliances based on whether the perceived size difference is within an allowable range or whether the perceived size difference is outside the allowable range.

[0375] When multiple placement areas are recognized, the control unit (150) may recognize an optimal placement area among the multiple placement areas where an outlet exists, and recognize the recognized optimal placement area as the placement location of the recommended home appliance.

[0376] The control unit (150) may recognize distance information between each of the multiple possible placement areas and the outlet based on location information of the multiple possible placement areas and location information of the outlet, and may recognize ranking information for the multiple possible placement areas based on the recognized distance information, and may also control a user interface to display ranking information for the recognized multiple possible placement areas.

[0377] When placing existing home appliances in an indoor space, if operating specifications or the size of the indoor space need to be considered, the control unit (150) can recognize one or more recommended home appliances among a plurality of candidate home appliances based on the size information of the empty area, the operating specification information of the candidate home appliances, and the size information of the indoor space.

[0378] The control unit (150) recognizes optimal size information corresponding to the size information of an indoor area based on the recognized first priority being space utilization, assigns a first score to each of the multiple candidate home appliances based on the recognized optimal size information and the size information of the multiple candidate home appliances, recognizes a placeable area where each candidate home appliance can be placed based on the size information of the empty area and the size information of the multiple candidate home appliances, assigns a second score to the multiple candidate home appliances where the placeable area exists, assigns a third score corresponding to optimal performance to each of the recognized multiple candidate home appliances based on the location information of the placeable area of ​​each candidate home appliance and the operation specification information of each candidate home appliance, sums the first, second, and third scores for each candidate home appliance, determines the ranking of each candidate home appliance based on the total score for each candidate home appliance, and recognizes one or more candidate home appliances from the first ranking to the pre-set ranking among the rankings of the candidate home appliances as recommended home appliances.

[0379] Here, the empty area may include an area where the first and second objects are not provided, and an area where the home appliance has been deleted.

[0380] The control unit (150) may also recognize one or more candidate home appliances with a score greater than or equal to the reference score as recommended home appliances.

[0381] The control unit (150) recognizes the heating and cooling capacity corresponding to the size information of the indoor space based on the recognized first priority being energy efficiency, and can recognize one or more candidate appliances having the recognized heating and cooling capacity as recommended appliances.

[0382] When multiple candidate home appliances are recognized as recommended home appliances, the control unit (150) recognizes the similarity between the cooling and heating capacity of the multiple candidate home appliances and the recognized cooling and heating capacity as a first score, and among the recognized first scores, the candidate home appliance having a score greater than or equal to the reference score can be recognized as a recommended home appliance.

[0383] When multiple candidate home appliances are recognized as recommended home appliances, the control unit (150) recognizes the similarity between the cooling and heating capacity of the multiple candidate home appliances and the recognized cooling and heating capacity as a first score, determines the ranking of the multiple candidate home appliances based on the first scores of the multiple candidate home appliances, and can recognize the candidate home appliances from the first ranking up to the preset ranking as recommended home appliances.

[0384] The ranking of candidate home appliances can be the ranking of recommended home appliances.

[0385] When the control unit (150) determines the ranking of candidate home appliances, the higher the first score, the higher the ranking can be determined.

[0386] As illustrated in FIG. 18, the control unit (150) may determine a ranking between one or more recommended appliances and existing appliances based on a first priority, and control the user interface (110) to display identification information of existing appliances together with identification information of recommended appliances based on the ranking of existing appliances being determined to be higher than a preset ranking.

[0387] The control unit (150) can control the user interface to display a simulation image of an indoor space in the first area (m1) of the user interface, and to display identification information and detailed information of recommended home appliances and identification information and detailed information of existing home appliances in the second area (m2) of the user interface, while also displaying the ranking.

[0388] The control unit (150) receives selection information of a home appliance received through the input unit (111), recognizes identification information of the home appliance based on the received selection information of the home appliance, and can also control the user interface (110) to display a simulation image of an indoor space based on the recognized identification information of the home appliance.

[0389] I will explain the case where other types of home appliances related to existing appliances are recommended.

[0390] The control unit (150) receives selection information of an existing home appliance received through the input unit (111), recognizes identification information of the existing home appliance based on the received selection information of the existing home appliance, and can recognize identification information of other types of recommended home appliances related to the existing home appliance based on the recognized identification information of the existing home appliance.

[0391] The control unit (150) can recognize a first priority selected by the user based on priority information received through the input unit (111), and recognize a recommended home appliance based on the recognized first priority and identification information of the recommended home appliance.

[0392] The configuration of the control unit (150) for recognizing a recommended home appliance of a different type from the existing home appliance based on the first priority is the same as the configuration of the control unit (150) for recognizing a recommended home appliance of the same type as the existing home appliance based on the first priority, so the description is omitted.

[0393] <Priority Control - Multiple Priorities>

[0394] Hereinafter, the configuration of the control unit is described in the case where two or more priority information is received for each type of home appliance. This embodiment describes the configuration of the control unit when receiving first, second, and third priorities for each existing home appliance.

[0395] The priority for reconfiguring home appliances can also be pre-set and stored for each type of appliance during the design and manufacturing process.

[0396] When at least two priorities are received for each home appliance through the input unit (111), the control unit (150) can store two or more priorities by matching them with the identification information of the home appliance.

[0397] Among the priorities selected by the user, the highest priority is listed as the 1st priority, the second highest priority is listed as the 2nd priority, and the third highest priority is listed as the 3rd priority.

[0398] The control unit (150) receives selection information of an existing home appliance received through the input unit (111), recognizes identification information of the existing home appliance based on the received selection information of the existing home appliance, and recognizes identification information of home appliances of the same type as the existing home appliance based on identification information of a plurality of new home appliances and the recognized identification information of the existing home appliance. Hereinafter, home appliances of the same type as the existing home appliance are listed as candidate home appliances.

[0399] The identification information of multiple new home appliances may be the identification information of home appliances stored in the memory (152) of the server (200) or the user device.

[0400] The control unit (150) recognizes a first priority, a second priority, and a third priority for the reconfiguration of an existing home appliance based on priority information received through the input unit (111).

[0401] The control unit (150) can recognize identification information of multiple first recommended home appliances among multiple candidate home appliances based on a first priority. More specifically, the control unit (150) assigns a first score to each of the multiple candidate home appliances based on the first priority and determines the ranking of the multiple candidate home appliances based on the first scores of the multiple candidate home appliances.

[0402] The first priority can be any one of interior harmony, space utilization, and energy efficiency.

[0403] Determining the first score of multiple candidate home appliances based on the first priority may be the same as determining the total score of multiple candidate home appliances based on the first priority among the previously described configurations.

[0404] Determining the first score of multiple candidate home appliances based on the first priority may be equivalent to recognizing the similarity of multiple candidate home appliances based on the first priority among the previously described configurations. An explanation of this will be omitted.

[0405] The control unit (150) can recognize multiple candidate home appliances from the first rank to the preset rank as multiple first recommended home appliances based on the ranking of multiple candidate home appliances.

[0406] The control unit (150) can also recognize multiple candidate home appliances having a first score greater than or equal to a first reference score as multiple first recommended home appliances.

[0407] The control unit (150) can recognize identification information of multiple second recommended home appliances among multiple first recommended home appliances based on a second priority. More specifically, the control unit (150) assigns a second score to each of the multiple first recommended home appliances based on the second priority and determines the ranking of the multiple first recommended home appliances based on the second scores of the multiple first recommended home appliances.

[0408] The second priority can be one of two conditions excluding the first priority.

[0409] Determining the second score of multiple first recommended home appliances based on the second priority may be the same as determining the total score of multiple candidate home appliances based on the first priority among the previously described configurations.

[0410] Determining the second scores of multiple first-recommended home appliances based on the second priority may be equivalent to recognizing the similarity of multiple candidate home appliances based on the first priority among the previously described configurations. An explanation of this will be omitted.

[0411] The control unit (150) can recognize multiple first recommended home appliances from first rank to preset ranks as multiple second recommended home appliances based on the ranking of multiple first recommended home appliances.

[0412] The control unit (150) can also recognize multiple first recommended home appliances having a second score greater than or equal to a second reference score as multiple second recommended home appliances.

[0413] The control unit (150) can recognize identification information of multiple third recommended home appliances among multiple second recommended home appliances based on the third priority. More specifically, the control unit (150) assigns a third score to each of the multiple second recommended home appliances based on the third priority and determines the ranking of the multiple second recommended home appliances based on the third scores of the multiple second recommended home appliances.

[0414] The third priority may be one condition excluding the first priority and the second priority.

[0415] Determining the third score of multiple second-recommended home appliances based on the third priority may be the same as determining the total score of multiple candidate home appliances based on the first priority among the previously described configurations.

[0416] Determining the third score of multiple second-recommended home appliances based on the third priority may be equivalent to recognizing the similarity of multiple candidate home appliances based on the first priority among the previously described configurations. An explanation regarding this will be omitted.

[0417] The control unit (150) can recognize one or more second recommended home appliances from first rank to preset rank as third recommended home appliances based on the ranking of multiple second recommended home appliances.

[0418] The control unit (150) may also recognize one or more second recommended home appliances having a third score greater than or equal to the third standard score as third recommended home appliances.

[0419] That is, the control unit (150) can recognize a first recommended home appliance among a plurality of candidate home appliances based on a first priority, recognize a second recommended home appliance among the first recommended home appliances based on a second priority, and recognize a third recommended home appliance among the second recommended home appliances based on a third priority.

[0420] The control unit (150) can control the user interface (110) to display the third recommended home appliances as recommended home appliances of the existing home appliances.

[0421] For example, we will explain the case where the recommended appliance is a refrigerator, the first priority is interior harmony, and the second priority is space utilization.

[0422] The control unit (150) can assign a first score to a plurality of candidate home appliances based on the appearance information of a plurality of candidate home appliances and the environmental element information of an indoor space, recognize a plurality of first recommended home appliances from first to fifth rank based on the first score of the plurality of candidate home appliances, assign a second score to a plurality of first recommended home appliances based on the size information of the empty area of ​​an indoor space and the size information of the plurality of first recommended home appliances, recognize a plurality of second recommended home appliances from first to third rank based on the second score of the plurality of first recommended home appliances, recognize the recognized second recommended home appliances from first to third rank as recommended home appliances for existing home appliances, and control a user interface to display identification information of the recognized second recommended home appliances.

[0423] The control unit (150) determines a first score for each of a plurality of candidate home appliances based on a first priority, determines a second score for each of a plurality of candidate home appliances based on a second priority, determines a third score for each of a plurality of candidate home appliances based on a third priority, sums the first, second, and third scores of each candidate home appliance to recognize the total score of each candidate home appliance, and it is also possible to recognize a recommended home appliance based on the total score of the candidate home appliances.

[0424] As illustrated in FIG. 19, the control unit (150) recognizes selection information of a recommended home appliance received through the input unit (111) and controls the user interface (110) to display a simulation image of a recommended home appliance placed in a first area (m1) of the user interface based on identification information of the recognized recommended home appliance, and also controls the user interface (110) to display detailed information about the recommended home appliance and a save button image in a second area (m2) of the user interface.

[0425] For example, detailed information of a recommended home appliance may include at least one of identification information, appearance information, operation specification information, sales location information, price information, and placement suitability information with an indoor space.

[0426] The control unit (150) may include at least one processor (151) for controlling the operation of the user device (100) and at least one memory (152) for storing a program and data for controlling the operation of the user device (100).

[0427] At least one processor (151) may include an algorithm for controlling the operation of components within the user device (100), at least one memory for storing data in the form of a program, and one or more processor chips that perform the aforementioned operation using the data stored in at least one memory, or one or more processing cores.

[0428] At least one processor (151) can process various data and various signals using instructions, data, programs and / or software stored in memory (152).

[0429] At least one processor (151) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator.

[0430] The memory (152) can store priority information for each home appliance for reconfiguring the home appliance.

[0431] The memory (152) can store identification information and detailed information of the recommended home appliance selected by the user.

[0432] The memory (152) can store information about the user's style and can store identification information of home appliances by style for interior design.

[0433] The memory (152) can store information about the first, second, and third reference scores and can store information about the reference similarity.

[0434] The memory (152) can store identification information and appearance information of multiple new home appliances received from the server (200).

[0435] The memory (152) can store data required for various embodiments.

[0436] The memory (152) may be implemented in the form of a memory embedded in the user device (100) or in the form of a memory that can be attached to the user device (100) depending on the purpose of data storage. For example, data for operating the user device (100) may be stored in a memory embedded in the user device (100), and data for the expansion function of the user device (100) may be stored in a memory that can be attached to the user device (100).

[0437] Meanwhile, the memory embedded in the user device (100) may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD).

[0438] In addition, the memory that can be attached to and detached from the user device (100) may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.) or an external memory that can be connected to a USB port (e.g., USB memory), but is not limited thereto.

[0439] The memory (152) may include one or more memory chips or one or more memory blocks.

[0440] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0441] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives) are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0442] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0443] At least one component may be added or removed in response to the performance of the components of the user device (100) illustrated in FIG. 2. Additionally, it will be readily understood by those skilled in the art that the relative positions of the components may be changed in response to the performance or structure of the user device (100).

[0444] Each component shown in Figure 2 refers to a software and / or hardware component such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0445] FIG. 20 is a control flowchart of a user device according to an embodiment.

[0446] The user device can execute an application (161) based on receiving an execution command for the application through the input unit (111), and can display execution information of the application through the display unit (112) while the application is being executed.

[0447] Here, the application may include an application for interior design services for indoor spaces.

[0448] When a scan command of an indoor space is received through the input unit (111) (162), the user device can control the activation of the image sensor (120) and the distance sensor (130).

[0449] The user device can acquire indoor space image information using an image sensor (120) (163) and recognize distance information with indoor objects using a distance sensor (130) (164).

[0450] The indoor objects include a first object including a home appliance, and a second object including static and dynamic elements excluding the home appliance.

[0451] Static elements may include at least one of a ceiling, wall, floor surface, stairs, door, and window.

[0452] Dynamic elements may include furniture, lighting, and decorative items. Additionally, the first object, the home appliance, may be included in the dynamic elements.

[0453] The user device can recognize information about the indoor space (165) based on indoor space image information acquired by the image sensor (120) and distance information with indoor objects detected by the distance sensor (130), and can recognize environmental element information of the indoor space (166).

[0454] Information on environmental elements of an indoor space may include information on the components of the indoor space and indoor objects.

[0455] More specifically, the user device can recognize the size information and shape information of the indoor space based on the distance information with indoor objects received from the distance sensor (130), and can recognize the location information, size information and shape information of the indoor objects based on the distance information with indoor objects received from the distance sensor (130).

[0456] The user device recognizes an empty area in an indoor space based on the size information of the indoor space, the shape information of the indoor space, the location information of indoor objects, the size information of indoor objects, and the shape information of indoor objects, and can recognize the size information and shape information of the recognized empty area. Here, the empty area may be an area in the indoor space where no indoor object is placed.

[0457] The user device can recognize shape information, size information of empty areas in the indoor space and shape information and size information of indoor objects based on indoor space image information acquired by the image sensor (120).

[0458] The user device can recognize a representative shape of the indoor space based on shape information of the indoor space and shape information of indoor objects, and can recognize color information and material information of indoor objects in the indoor space based on image information of the indoor space.

[0459] More specifically, the user device can recognize the primary and secondary colors of the indoor space based on indoor space image information and a pre-stored reference color table.

[0460] The dominant color can be the color that occupies the largest proportion in the image of an indoor space.

[0461] The secondary color can be the color that accounts for the second-largest proportion in an indoor space image.

[0462] The user device can recognize reflection information of multiple indoor objects based on indoor space image information, and recognize material information of each of the multiple indoor objects based on the reflection information of the multiple indoor objects.

[0463] Material information may include information regarding glossy and matte materials.

[0464] The user device recognizes the color distribution and clarity of multiple indoor objects based on indoor space image information, and it is also possible to recognize the material information of each of the multiple indoor objects based on the color distribution and clarity of the multiple indoor objects.

[0465] The user device can store material information for each of multiple indoor objects by matching it with an image of each of the multiple indoor objects.

[0466] The user device recognizes the number of indoor objects made of glossy material and the number of indoor objects made of matte material, compares the recognized number of indoor objects made of glossy material and the number of indoor objects made of matte material to recognize the material with the larger number, and can recognize the recognized material as the representative material of the indoor space.

[0467] The user device processes indoor space image information and recognizes pixels (EP) corresponding to the edges in the processed indoor space image, recognizes multiple indoor objects based on the recognized pixels (EP), and can recognize edge information of each indoor object based on the recognized pixels (EP).

[0468] The user device recognizes a first object among a plurality of recognized indoor objects based on the boundary information of the indoor object for each recognized indoor object and the boundary information of the first objects stored in memory (152) (167), and can recognize identification information of the recognized first object and operation specification information of the first object based on the boundary information of the recognized first object and the boundary information of the first objects stored in memory (152).

[0469] The identification information of the first object may include the name information of the existing home appliance, the model information of the existing home appliance, and the serial information of the existing home appliance.

[0470] The name information of the first object may be information corresponding to the type of home appliance, such as a refrigerator, washing machine, air conditioner, etc.

[0471] The user device can display identification information of the first object placed in the indoor space through the display unit (112) (168). That is, the user device can display identification information of the existing home appliance placed in the indoor space through the display unit (112).

[0472] The user device may also display multiple first objects as a list based on the recognition of multiple first objects.

[0473] The user device can display an image of an indoor space as a three-dimensional simulation image based on a display command of a simulation image received through the input unit (111).

[0474] The user device can display an image of an indoor space as an augmented reality image based on a display command of a simulation image received through the input unit (111).

[0475] Based on a display command of a simulation image received through an input unit (111), the user device can display an indoor space image as an augmented reality image in a first area of ​​the display unit and display a list of multiple first objects in a second area of ​​the display unit.

[0476] The user device recognizes the suitability of placement of the first object based on environmental element information of the indoor space, identification information of the first object, location information of the first object, and operation specification information of the first object (169), and recognizes that the home appliance needs to be relocated based on information of unsuitability of placement of the first object.

[0477] When the control unit (150) recognizes that the first object needs to be relocated (170), it recognizes the optimal position information of the first object based on the environmental element information of the indoor space and the operation specification information of the first object (171), generates new indoor space image information based on the recognized optimal position information and the image information of the first object, and can display the new indoor space image information as a simulation image (172).

[0478] The user device can recognize psychological environmental element information among the environmental element information of the indoor space, recommend new home appliances related to existing home appliances based on the recognized psychological environmental element information, and display identification information of the recommended home appliances.

[0479] Information on psychological environmental factors may include information on components of the indoor space, such as shape, material, and color, and information on components of indoor objects, such as shape, material, and color.

[0480] Information on psychological environmental factors may include information on components such as the representative shape, representative material, primary color, and secondary color of the indoor space.

[0481] The user device recognizes identification information of a recommended home appliance received through the input unit (111), recognizes optimal location information of the recognized recommended home appliance based on the identification information, appearance information, and operation specification information of the recognized recommended home appliance, generates a new indoor space image based on the recognized optimal location information and the identification information of the recognized recommended home appliance, and can display the generated new indoor space image as a simulation image.

[0482] FIG. 21 is a control flowchart for recommending and displaying home appliances of a user device according to an embodiment.

[0483] The user device can receive priority information regarding the priority of the reconfiguration conditions of home appliances in the indoor space through the input unit (111) (181).

[0484] The user device can recognize a first priority, a second priority, and a third priority for the reconfiguration of an existing home appliance based on received priority information.

[0485] The conditions for reconfiguring home appliances within an indoor space are conditions for reconfiguring existing or recommended home appliances within the indoor space, and may be conditions for receiving recommendations for the placement locations of home appliances or for receiving recommendations for new home appliances.

[0486] Reconfiguration conditions may include space utilization, interior harmony, and energy efficiency, and may additionally include price.

[0487] The configuration of the control unit for receiving the first, second, and third priorities for each existing home appliance is described. The highest priority among the priorities selected by the user is listed as the first priority, the second highest priority is listed as the second priority, and the third highest priority is listed as the third priority.

[0488] The user device receives selection information of an existing home appliance received through the input unit (111), recognizes identification information of an existing home appliance based on the received selection information of an existing home appliance, and can select the type of home appliance to be recommended based on the recognized identification information of an existing home appliance (182).

[0489] The types of home appliances to be recommended may include types identical to existing home appliances or types different from existing home appliances.

[0490] The type of home appliance to be recommended can also be received through the input unit (111).

[0491] The user device can also select the type of home appliance to be recommended based on identification information of existing home appliances in the indoor space and information of environmental factors in the indoor space (182).

[0492] For example, a user device recognizes the types of existing home appliances placed in an indoor space, recognizes whether there are additional home appliances needed in the indoor space based on the recognized types of existing home appliances, and can recognize a different type of home appliance to be recommended based on the existence of additional home appliances.

[0493] The user device recognizes the need to change existing home appliances based on identification information of existing home appliances placed in the indoor space and information on environmental elements of the indoor space, and based on the recognition that the existing home appliances need to be changed, it can recognize the home appliance to be recommended as a home appliance of the same type.

[0494] The user device can recognize multiple candidate appliances among multiple new appliances based on the recognized type (183).

[0495] The user device can recognize a recommended appliance among multiple candidate appliances based on priority information (184).

[0496] Explain the recognition configuration of recommended home appliances in more detail.

[0497] The user device can recognize identification information of multiple first recommended home appliances among multiple candidate home appliances based on the first priority.

[0498] More specifically, the user device assigns a first score to each of the multiple candidate home appliances based on the identification information, appearance information, operation specification information, indoor space environmental factor information, and first priority of the multiple candidate home appliances, and determines the ranking of the multiple candidate home appliances based on the first scores of the multiple candidate home appliances.

[0499] Based on the ranking of multiple candidate home appliances, the user device can recognize multiple candidate home appliances from the first rank to a preset rank as multiple first recommended home appliances.

[0500] The user device can recognize identification information of multiple second recommended home appliances among multiple first recommended home appliances based on the second priority.

[0501] More specifically, the user device assigns a second score to a plurality of first recommended home appliances based on identification information, appearance information, operation specification information, indoor space environmental factor information, and a second priority of a plurality of first recommended home appliances, and determines the ranking of a plurality of second recommended home appliances based on the second scores of the plurality of first recommended home appliances.

[0502] Based on the rankings of multiple first recommended home appliances, the user device can recognize multiple first recommended home appliances from the first rank to a preset rank as multiple second recommended home appliances.

[0503] The user device can recognize identification information of multiple third-recommended home appliances among multiple second-recommended home appliances based on the third priority.

[0504] More specifically, the user device assigns a third score to each of the multiple second recommended home appliances based on the identification information, appearance information, operation specification information, indoor space environmental factor information, and third priority of the multiple second recommended home appliances, and determines the ranking of the multiple second recommended home appliances based on the third scores of the multiple second recommended home appliances.

[0505] The user device can recognize one or more second recommended home appliances, ranging from first rank to a preset rank, as third recommended home appliances based on the rankings of multiple second recommended home appliances.

[0506] That is, the user device recognizes the third recommended home appliances from the first rank to the preset rank as recommended home appliances of the existing home appliance, recognizes the identification information of the recognized recommended home appliances from the first rank to the preset rank, and can display the identification information of the recognized recommended home appliances from the first rank to the preset rank on the display unit (185).

[0507] The user device recognizes selection information of recommended home appliances received through the input unit (111) (186), and can recognize identification information of recommended home appliances based on the recognized selection information.

[0508] The user device can recognize the optimal location information of the recommended appliance based on the identification information of the recognized recommended appliance, the appearance information of the recommended appliance, the operation specification information of the recommended appliance, and the environmental element information of the indoor space, and can generate an image of the indoor space where the recommended appliance is placed based on the recognized optimal location information and display the generated image of the indoor space as a simulation image (187).

[0509] The user device can display a simulation image in which a recommended home appliance is placed in the first area (m1) of the user interface and display detailed information about the recommended home appliance and a save button image in the second area (m2) of the user interface.

[0510] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. The instructions may be stored in the form of program code and, when executed by a processor, may generate a program module to perform the operation of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0511] Computer-readable recording media include all types of recording media that store instructions that can be decoded by a computer. Examples include ROM (Read Only Memory), RAM (Random Access Memory), magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0512] As described above, the disclosed embodiments have been explained with reference to the attached drawings. Those skilled in the art will understand that the present invention may be practiced in forms different from the disclosed embodiments without changing the technical spirit or essential features of the invention. The disclosed embodiments are illustrative and should not be interpreted restrictively.

Claims

1. An image sensor for acquiring an indoor space image of an indoor space; A distance sensor that detects the distance to a plurality of indoor objects within the above indoor space; A processor that identifies components of an indoor space based on at least one of the distances between the indoor space image and the plurality of indoor objects, acquires spatial information corresponding to the identified components of the indoor space based on the identified components of the indoor space, and reconstructs the interior design of the indoor space based on the acquired spatial information; and A user device comprising a display unit that displays the interior design of the above-mentioned indoor space as a simulation image.

2. In paragraph 1, the processor, A user device that identifies at least one existing home appliance included in the plurality of indoor objects, obtains identification information of the at least one existing home appliance based on the identified at least one existing home appliance, identifies operation specification information of the at least one existing home appliance based on the obtained identification information of the at least one existing home appliance, identifies an optimal location of the at least one existing home appliance based on at least one of the spatial information, the identification information of the at least one existing home appliance, and the operation specification information of the at least one existing home appliance, and reconstructs the interior design of the indoor space based on the identified optimal location.

3. In paragraph 1, the processor, A user device that identifies at least one existing home appliance included in the plurality of indoor objects and controls the display unit to display the identified at least one existing home appliance in one area of ​​the display unit based on the identification of the at least one existing home appliance included in the plurality of indoor objects.

4. In paragraph 1, the processor, A user device that identifies boundary information of a plurality of indoor objects based on the indoor space image, identifies a first object including at least one existing home appliance and a second object other than the first object based on the identified boundary information of the plurality of indoor objects, and controls the display unit so that the first object and the second object are displayed in a different manner in the indoor space image.

5. In paragraph 1, the processor, A user device that identifies at least one existing home appliance included in the plurality of indoor objects, obtains identification information of the at least one existing home appliance based on the identified at least one existing home appliance, identifies at least one recommended home appliance among the at least one new home appliances corresponding to the indoor space based on the obtained identification information of the at least one existing home appliance and identification information of at least one new home appliance stored in advance, and controls the display unit to display the identification information of the identified at least one recommended home appliance.

6. In paragraph 5, the above processor, A user device that identifies the at least one recommended appliance based on whether the at least one new appliance is of the same type as or interacts with the at least one existing appliance.

7. In paragraph 5, the above processor, A user device that identifies the optimal location of the at least one recommended home appliance based on identification information of the at least one recommended home appliance and the spatial information, and reconstructs the interior design of the indoor space based on the identified optimal location of the at least one recommended home appliance.

8. In Paragraph 5, It further includes an input unit that receives user input, and The above processor is a user device that reconstructs the interior design of the indoor space based on the selection of at least one recommended home appliance from the input unit.

9. In Paragraph 1, The above spatial information includes information about a first object including at least one existing home appliance existing within the indoor space, information about a second object other than the first object, and psychological element information about the first and second objects. The above psychological element information is a user device comprising at least one of the brand information, shape information, size information, material information, and color information of the first and second objects.

10. In Paragraph 1, It further includes an input unit that receives priority information regarding the reconstruction of the interior design of the above indoor space, and The processor determines the rank of at least one recommended home appliance based on priority information received through the input unit, the indoor space information, and identification information of at least one existing home appliance existing within the indoor space, and controls the display unit to display the identification information of the at least one recommended home appliance based on the determined rank. The above priority information is a user device that includes at least one of information regarding interior harmony, price, space utilization, and energy efficiency of a home appliance.

11. In Clause 10, the above processor, A user device that determines a ranking between at least one existing home appliance and at least one recommended home appliance based on priority information received through the input unit, spatial information, and identification information of at least one existing home appliance.

12. A step of acquiring an indoor space image of the indoor space; A step of identifying the distance to a plurality of indoor objects within the above indoor space; A step of identifying components of the indoor space based on at least one of the indoor space image and the distance to the plurality of indoor objects; A step of obtaining spatial information corresponding to the components of the identified indoor space based on the components of the identified indoor space; A step of reconstructing the interior design of the indoor space based on the spatial information obtained above; and A method for controlling a user device comprising the step of displaying the interior design of the above indoor space as a simulation video.

13. In Clause 12, the step of reconstructing the interior design of the indoor space is, A step of identifying at least one existing home appliance included in the plurality of indoor objects, A step of obtaining identification information of at least one existing home appliance based on the at least one existing home appliance identified above, a step of identifying operation specification information of the existing home appliance based on the obtained identification information of at least one existing home appliance, A step of identifying the optimal location of the at least one existing home appliance based on at least one of the spatial information, identification information of the at least one existing home appliance, and operation specification information of the at least one existing home appliance, and A method for controlling a user device, comprising the step of reconstructing the interior design of the indoor space based on the optimal location of at least one existing home appliance identified above.

14. In Clause 12, the step of reconstructing the interior design of the indoor space is, A step of identifying at least one existing home appliance included in the plurality of indoor objects, A step of obtaining identification information of at least one existing home appliance based on at least one existing home appliance identified above, A method for controlling a user device comprising the step of identifying at least one recommended appliance corresponding to the indoor space among the at least one new appliances based on identification information of at least one existing appliance obtained above and identification information of at least one new appliance stored in advance.

15. In paragraph 14, the step of identifying at least one recommended home appliance is, A method for controlling a user device comprising the step of identifying at least one recommended appliance based on whether the at least one new appliance is of the same type as or interacts with the at least one existing appliance.

Images